-- Logs begin at Sun 2022-09-25 00:10:28 UTC, end at Sun 2022-09-25 01:17:39 UTC. -- Sep 25 00:46:52.479136 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 25 00:46:52.488422 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 25 00:46:52 2022] *** Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: nodename: dv-cinder-master-3-iscsi Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: machine: x86_64 Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: clock source: unix Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: pcre jit disabled Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: detected number of CPU cores: 4 Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: current working directory: / Sep 25 00:46:52.490408 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: detected binary path: /usr/bin/uwsgi-core Sep 25 00:46:52.491124 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: your processes number limit is 47680 Sep 25 00:46:52.491124 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: your memory page size is 4096 bytes Sep 25 00:46:52.491124 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: detected max file descriptor number: 2048 Sep 25 00:46:52.491124 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: lock engine: pthread robust mutexes Sep 25 00:46:52.491124 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: thunder lock: enabled Sep 25 00:46:52.491124 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 25 00:46:52.491124 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 25 00:46:52.532113 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: Python main interpreter initialized at 0x55787c5c89c0 Sep 25 00:46:52.532113 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: python threads support enabled Sep 25 00:46:52.532113 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: your server socket listen backlog is limited to 100 connections Sep 25 00:46:52.532113 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: your mercy for graceful operations on workers is 90 seconds Sep 25 00:46:52.532564 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: mapped 403077 bytes (393 KB) for 2 cores Sep 25 00:46:52.532564 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: *** Operational MODE: preforking *** Sep 25 00:46:52.532855 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: *** uWSGI is running in multiple interpreter mode *** Sep 25 00:46:52.532855 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: spawned uWSGI master process (pid: 105409) Sep 25 00:46:52.533077 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 25 00:46:52.534290 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: spawned uWSGI worker 1 (pid: 105410, cores: 1) Sep 25 00:46:52.534290 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: spawned uWSGI worker 2 (pid: 105411, cores: 1) Sep 25 00:46:52.534290 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 25 00:46:53.205106 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 25 00:46:53.208755 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 25 00:46:54.314128 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:46:54.314128 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: from cryptography.utils import int_from_bytes Sep 25 00:46:54.314789 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:46:54.314789 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: from cryptography.utils import int_from_bytes Sep 25 00:46:54.315202 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:46:54.315202 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: from cryptography.utils import int_from_bytes Sep 25 00:46:54.315365 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:46:54.315365 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: from cryptography.utils import int_from_bytes Sep 25 00:46:56.826972 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 25 00:46:56.827508 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 25 00:46:56.827662 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 25 00:46:56.828022 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 25 00:46:56.828278 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 25 00:46:56.828579 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 25 00:46:56.828977 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.829065 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 25 00:46:56.829158 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.829311 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.829403 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 25 00:46:56.829634 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.829729 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 25 00:46:56.829966 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.830070 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 25 00:46:56.830276 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.830378 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.830598 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.830697 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.830913 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.831008 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.831240 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.831342 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.831500 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.831588 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.831816 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.831905 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.832138 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.832237 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.832459 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.832554 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.832766 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.832866 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.833021 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.833115 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.833358 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.833470 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.833682 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.833783 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.834002 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.834103 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.834306 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.834407 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.834607 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.834801 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.834914 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.835969 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836105 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836215 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836352 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836438 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836563 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836563 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836785 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836887 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.836981 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.837068 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.837155 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.837235 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.837371 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.837466 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.837730 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.837832 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.838040 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.838266 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.838452 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.838593 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.838770 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.838899 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.839087 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.839263 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.839403 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.839519 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.839815 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.839909 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.840126 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.840239 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.840449 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.840547 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.840784 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.840897 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.841066 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.841155 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.841399 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.841507 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.841730 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.841817 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.842052 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.842143 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.842371 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.842472 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.842693 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.842780 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.842996 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.843083 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.843305 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.843415 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.843617 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.843724 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.843935 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.844029 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.844239 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.844327 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.844543 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.844635 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.844794 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.844921 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.845100 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.845199 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.845449 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.845551 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.845761 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.845859 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.846065 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.846165 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.846311 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.846405 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.846557 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.846674 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.846878 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.846975 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.847198 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.847294 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.847508 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.847611 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.847770 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.847864 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.848016 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.848111 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.848330 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.848420 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.848650 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.848745 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.848957 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.849053 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.849287 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.849377 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.849585 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.849675 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.849891 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.849991 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.850194 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.850282 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.850497 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.850584 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.850805 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.850938 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.851111 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.851226 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.851427 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.851516 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.52 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.851737 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.851835 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.52 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.852043 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.852130 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.852348 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.852434 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.852664 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.852753 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.853689 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.52 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.853793 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.854027 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.854249 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.52 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.854356 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.854476 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.854648 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.854819 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.854987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.855174 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.855283 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.855451 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.855576 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.855773 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.855913 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.856096 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.856254 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.856403 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.856492 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.856709 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.856799 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.857021 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.857111 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.857364 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.857464 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.857670 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.857763 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.857979 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.858068 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.858227 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.858315 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.858528 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.858621 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.858831 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.858920 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.859149 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.859243 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.859453 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.859547 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.860012 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.860109 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.860316 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.860402 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.860616 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.860703 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.860918 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.861050 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.861218 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.861330 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.861549 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.861650 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.861850 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.861936 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.862154 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.862243 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.862457 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.862557 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.862760 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.862849 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.863060 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.863155 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.863360 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.863455 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.863660 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.863779 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.863974 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.864076 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.864275 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.864375 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.864577 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.864670 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.864880 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.865050 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.865183 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.865317 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.865513 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.865613 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.865829 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.865920 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.866133 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.866229 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.866434 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.866524 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.866752 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.866846 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.867053 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.867148 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.867351 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.867437 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.867665 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.867752 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.867965 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.868064 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.868267 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.868361 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.868619 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.869233 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.869469 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.869718 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.869814 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.870057 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.870151 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.870358 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.870454 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.870660 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.870756 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.870960 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.871055 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.871264 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.871414 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.871570 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.871666 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.871871 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.871959 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.872175 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.872271 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.872476 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.872572 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.872777 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.872865 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.873074 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.873163 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.873410 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.873513 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.873732 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.873829 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.874249 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.874356 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.874588 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.874677 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.874893 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.874979 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.875214 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:46:56.875309 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.875533 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.875620 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.875851 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.875939 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.876182 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.876271 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.876495 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.876590 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.876807 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.876904 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.877120 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.877218 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.877467 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.877558 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.877777 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.877877 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.878095 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.878183 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.878409 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.878496 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.878719 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.878820 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.879030 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.879125 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.879342 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.879432 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.879657 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.879753 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.879968 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.880062 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.880295 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.880385 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.880606 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.880707 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.880920 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.881015 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.881235 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.881366 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.881566 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.881669 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.881881 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.881970 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.882207 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.882296 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.882521 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.882608 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.882826 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.882913 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.883135 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.883229 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.883481 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.883576 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.883792 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.883878 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.884155 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.884255 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.884442 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.884692 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.884781 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.885064 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.885152 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.885402 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.885511 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.885715 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.885805 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.886027 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.886121 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.886340 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.886438 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.886652 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.886743 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.886989 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.887076 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.887306 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.887395 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.887627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.887724 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.887939 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.888026 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.888246 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.888337 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.888555 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.888644 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.888864 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.888951 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.889170 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.889291 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.889507 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.889597 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.889815 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.889922 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.890124 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.890211 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.890431 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.890526 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.890738 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.890833 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.891041 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.891134 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.891348 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.891440 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.891655 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.891761 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.891961 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.892049 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.892263 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.892350 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.892567 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.892667 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.892869 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.892963 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.893174 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.893322 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.893499 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.893602 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.893805 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.893892 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.894119 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.894221 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.894423 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.894524 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.894723 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.894811 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.895040 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.895127 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.895347 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.895435 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.895652 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.895740 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.895959 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.896055 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.896263 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.896352 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.896569 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.896744 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.896880 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.896976 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.897181 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.897296 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.897525 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.897614 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.897835 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.897939 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.898141 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.898236 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.898460 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.898548 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.898764 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.898861 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.899068 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.899157 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.899377 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.899465 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.899685 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.899888 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.900180 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.900269 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.900566 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.900669 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.900901 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.901006 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.901222 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.901362 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.901556 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.901679 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.901867 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.901958 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.902172 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.902262 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.902477 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.902576 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.902783 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.902872 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.903093 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.903182 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.903399 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.903566 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.903706 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.903888 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.904009 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.904192 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.904315 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.904507 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.904645 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.904811 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.904938 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.905112 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.905245 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.905443 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.905573 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.905752 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.905878 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.906054 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.906182 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.906356 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.906524 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.906658 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.906832 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.906966 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.907176 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.907274 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.907513 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.907610 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.907836 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.907924 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.908157 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.908254 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.908460 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.908556 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.908763 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.908851 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.909074 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.909162 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.909419 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.909529 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.909734 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.909822 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.910048 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.910144 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.910370 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.910459 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.910682 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.910777 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.910991 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.911087 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.911299 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.911386 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.911605 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.911691 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.911915 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.912015 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.912236 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.912335 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.912527 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.912625 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.912842 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.912929 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.913152 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.913246 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.913494 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.913599 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.913802 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.913891 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.914125 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.914214 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.914435 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.914549 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.914752 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.914850 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.915052 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.915147 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.915881 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.915990 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.916196 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.916505 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.916592 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.916824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.916984 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.917136 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.917337 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.917481 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.917666 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.917793 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.917973 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.918117 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.918281 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.918427 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.918590 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.918744 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.918900 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.919051 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.919208 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.919368 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.919514 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.919702 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.919830 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.920038 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.920126 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.920349 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.920443 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.920662 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.920769 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.920987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.921077 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.921325 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.921417 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.921643 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.921739 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.921956 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.922045 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.922265 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.922352 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.922569 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.922658 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.922879 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.922966 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.923195 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.923284 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.923502 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.923589 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.923809 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.923903 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.924116 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.924203 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.924425 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.924514 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.924730 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.924816 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.925030 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.925126 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.925354 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.925444 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.925664 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.925761 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.925975 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.926111 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.926283 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.926377 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.926587 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.926674 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.926887 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.927028 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.927199 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.927296 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.927501 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.927589 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.927800 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.927897 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.928099 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.928186 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.928405 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.928495 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.928707 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.928795 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.929011 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.929099 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.929354 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.929444 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.929677 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.929768 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.929980 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.930067 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.930278 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.930365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.930597 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.930687 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.930902 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.930989 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.931295 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.931475 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.931605 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.931916 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.932020 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.932238 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.932335 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.932552 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.932655 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.932861 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.932950 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.933169 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.933279 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.933502 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.933593 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.933811 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.933900 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.934116 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.934202 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.934430 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.934520 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.934732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.934860 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.935036 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.935123 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.935341 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.935436 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.935645 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.935734 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.935948 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.936034 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.936257 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.936353 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.936563 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.936839 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.936929 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.937162 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.937294 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.937503 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.937591 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.937807 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.937896 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.938113 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.938204 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.938420 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.938519 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.938720 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.938811 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.939026 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.939127 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.939342 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.939438 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.939689 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.939911 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.940009 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.940244 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.940341 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.940559 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.940656 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.940876 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.940965 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.941178 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.941287 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.941512 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.941611 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.941819 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.941909 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.942129 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.942295 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.942432 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.942521 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.942735 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.942823 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.943039 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.943137 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.943346 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.943435 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.943644 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.943738 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.943948 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.944037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.944248 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.944336 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.944550 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.944645 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.944850 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.944987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.945157 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.945314 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.945482 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.945573 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.945784 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.945873 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.946084 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.946172 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.946387 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.946485 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.947214 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.947334 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.947558 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.947786 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.947883 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.948120 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.948218 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.948426 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.948523 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.948753 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.948842 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.949062 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.949151 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.949393 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.949486 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.949698 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.949797 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.950000 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.950096 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.950312 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.950400 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.950621 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.950775 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.950923 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.951024 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.951232 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.951322 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.951541 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.951645 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.951851 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.951946 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.952159 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.952264 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.952461 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.952566 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.952764 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.952852 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.953065 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.953154 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.953391 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.953495 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.953698 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.953787 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.953999 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.954093 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.954298 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.954388 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.954603 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.954691 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.954907 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.954995 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.955209 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.955323 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.955511 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.955599 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.955810 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.955898 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.956115 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.956205 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.956413 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.956510 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.956712 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.956807 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.957014 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.957102 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.957340 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.957433 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.957643 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.957750 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.957942 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.958031 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.958243 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.958337 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.958542 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.958640 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.958846 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.958935 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.959143 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.959240 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.959453 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.959550 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.959753 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.959841 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.960052 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.960140 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.960349 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.960449 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.960654 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.960750 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.960958 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.961047 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.961283 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.961375 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.961589 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.961677 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.961895 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.961987 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.962214 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.962332 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.962704 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.962792 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.963043 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.963145 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.963352 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.963451 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.963661 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.963767 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.963968 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.964131 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.964266 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.964357 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.964570 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.964659 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.964897 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.964996 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.965209 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.965328 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.965542 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.965632 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.965848 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.965945 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.966152 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.966244 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.966460 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.966592 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.966762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.966860 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.967064 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.967153 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.967370 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.967508 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.967670 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.967773 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.967971 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.968059 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.968289 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.968378 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.968595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.968684 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.968895 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.968983 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.969198 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.969305 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.969523 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.969622 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.969858 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.969945 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.970157 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.970244 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.970464 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.970553 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.970769 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.970902 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.971093 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.971181 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.971407 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.971507 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.971713 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.971800 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.972022 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.972110 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.972335 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.972432 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.972637 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.972726 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.972946 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.973042 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.973252 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.973400 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.973585 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.973673 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.973887 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.973988 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.974188 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.974276 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.974495 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.974582 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.974803 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.974899 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.975112 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.975201 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.975424 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.975521 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.975738 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.975840 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.976050 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.976138 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.976349 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.976475 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.976658 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.976751 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.976964 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.977058 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.977281 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.977373 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.977592 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.977690 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.978421 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.978526 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.978725 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.978985 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.979128 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.979319 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.979442 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.979635 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.979760 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.979949 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.980057 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.980263 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.980493 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.980582 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.980808 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.980906 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.981121 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.981210 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.981458 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.981559 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.981775 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.981874 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.982087 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.982181 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.982405 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.982502 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.982721 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.982821 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.983028 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.983118 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.983335 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.983449 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.983641 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.983764 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.984072 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.984161 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.984381 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.984474 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.984937 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.985040 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.985245 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.985413 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.985577 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.985677 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.985886 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.985984 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.986193 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.986280 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.986500 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.986591 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.986804 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.986901 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.987124 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.987219 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.987430 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.987519 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.987755 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.987859 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.988104 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.988192 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.988406 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.988503 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.988710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.988797 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.989016 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.989162 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.989364 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.989503 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.989671 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.989820 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.989971 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.990145 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.990272 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.990471 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.990591 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.990800 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.990889 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.991109 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.991207 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.991425 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.991597 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.991737 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.991907 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.992063 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.992210 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.992390 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.992623 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.992725 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.992932 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.993027 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.993239 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.993385 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.993644 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.993781 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.994176 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.994267 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.994601 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.994689 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.994939 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.995029 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.995257 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.995344 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.995568 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.995656 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.995893 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.995983 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.996216 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.996312 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.996526 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.996614 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.996839 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.996932 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.997142 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.997230 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.997494 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.997598 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.997809 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.997898 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.998118 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.998205 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.998438 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.998535 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.998744 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.998832 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.999056 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.999144 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.999367 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.999464 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.999684 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.999781 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:56.999989 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.000078 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.000293 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.000390 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.000619 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.000708 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.000924 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.001012 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.001226 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.001383 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.001556 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.001646 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.001879 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.001971 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.002185 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.002276 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.002494 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.002582 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.002798 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.002885 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.003105 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.003204 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.003407 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.003496 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.003703 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.003791 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.004004 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.004093 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.004347 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.004437 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.004655 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.004760 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.004978 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.005074 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.005335 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.005427 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.005671 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.005773 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.005973 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.006061 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.006293 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.006389 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.006599 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.006697 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.006902 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.007046 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.007207 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.007296 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.007507 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.007604 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.007836 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.007936 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.008137 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.008225 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.008438 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.008527 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.008749 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.008837 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.009055 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.009745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.009928 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.010021 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.010302 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.010592 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.010682 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.010907 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.011004 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.011211 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.011300 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.011513 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.011601 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.011836 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.011937 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.012144 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.012234 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.012453 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.012546 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.012757 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.012873 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.013058 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.013145 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.013385 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.013485 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.013691 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.013780 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.013993 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.014082 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.014300 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.014390 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.014601 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.014691 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.014901 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.014997 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.015200 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.015290 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.015520 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.015610 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.015821 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.015914 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.016138 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.016235 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.016442 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.016530 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.016742 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.016830 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.017044 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.017145 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.017372 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.017464 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.017681 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.017770 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.017999 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.018101 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.018316 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.018405 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.018619 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.018707 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.018939 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.019184 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.019274 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.019515 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.019604 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.019830 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.019995 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.020146 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.020233 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.020457 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.020545 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.020766 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.020864 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.021078 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.021165 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.021405 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.021498 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.021717 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.021818 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.022026 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.022113 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.022334 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.022421 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.022663 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.022882 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.022972 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.023209 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.023304 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.023522 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.023620 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.023833 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.023922 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.024140 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.024235 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.024451 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.024548 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.024807 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.025085 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.025399 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.025587 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.025753 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.025893 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.026069 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.026159 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.026381 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.026479 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.026695 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.026784 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.027011 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.027101 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.027325 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.027417 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.027636 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.027735 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.027952 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.028079 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.028264 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.028362 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.028571 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.028659 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.028882 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.028978 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.029197 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.029314 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.029541 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.029630 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.029845 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.029933 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.030159 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.030250 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.030472 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.030561 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.030785 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.030872 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.031095 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.031186 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.031404 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.031500 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.031717 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.031922 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.032020 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.032238 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.032333 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.032571 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.032660 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.032898 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.032988 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.033207 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.033328 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.033554 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.033644 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.033871 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.033961 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.034189 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.034285 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.034495 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.034584 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.034800 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.034890 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.035112 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.035200 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.035422 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.035623 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.035721 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.035936 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.036025 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.036268 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.036363 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.036592 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.036682 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.036914 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.037003 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.037223 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.037408 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.037561 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.037779 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.037890 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.038132 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.038223 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.038453 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.038552 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.038777 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.038865 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.039084 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.039177 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.039385 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.039483 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.039706 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.039801 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.040014 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.040109 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.040315 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.040970 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.041210 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.041456 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.041543 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.041835 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.041938 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.042148 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.042370 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.042459 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.042721 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.042810 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.043039 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.043141 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.043343 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.043445 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.043656 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.043755 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.043960 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.044049 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.044281 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.044378 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.044586 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.044683 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.044892 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.044988 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.045194 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.045312 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.045535 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.045636 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.045842 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.045931 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.046146 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.046238 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.046447 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.046547 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.046752 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.046842 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.047056 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.047146 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.047359 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.047503 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.047669 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.047759 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.047968 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.048056 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.048277 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.048374 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.048583 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.048671 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.048887 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.048984 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.049189 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.049312 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.049524 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.049614 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.049828 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.049916 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.050129 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.050219 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.050431 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.050520 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.050733 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.050821 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.051042 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.051131 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.051343 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.051441 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.051651 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.051752 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.051953 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.052043 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.052261 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.052349 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.052617 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.052815 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.052906 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.053154 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.053242 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.053517 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.053618 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.053834 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.053930 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.054135 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.054224 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.054447 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.054535 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.054754 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.054845 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.055061 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.055160 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.055371 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.055472 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.055680 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.055777 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.056030 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.056166 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.056346 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.056658 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.056754 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.057461 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.057610 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.057698 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.057779 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.057941 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.058027 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.058108 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.058259 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.058351 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.058596 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.058694 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.058905 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.058994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.059212 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.059301 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.059518 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.059616 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.059825 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.059913 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.060139 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.060226 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.060451 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.060549 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.060760 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.060849 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.061068 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.061169 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.061401 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.061495 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.061712 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.061800 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.062022 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.062128 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.062330 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.062423 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.062633 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.062727 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.062938 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.063040 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.063246 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.063334 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.063550 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.063638 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.063858 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.063968 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.064164 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.064268 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.064471 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.064809 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.064906 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.065126 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.065215 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.065463 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.065566 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.065778 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.065880 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.066092 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.066182 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.066402 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.066506 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.066710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.066814 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.067019 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.067124 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.067322 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.067413 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.067634 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.067730 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.067958 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.068048 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.068269 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.068359 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.068572 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.068677 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.068922 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.069230 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.069383 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.069563 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.069664 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.069871 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.069960 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.070183 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.070323 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.070497 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.070587 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.070810 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.070899 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.071117 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.071206 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.071423 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.071513 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.072266 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.072415 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.072544 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.072823 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.072927 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.073298 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.073393 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.073617 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.073706 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.073949 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.074041 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.074255 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.074342 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.074563 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.074661 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.074866 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.074955 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.075168 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.075256 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.075475 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.075564 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.075780 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.075870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.076086 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.076181 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.076389 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.076476 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.076690 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.076785 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.077041 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.077148 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.077323 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.077433 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.077634 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.077811 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.077971 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.078060 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.078277 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.078376 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.078586 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.078674 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.078889 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.078977 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.079192 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.079290 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.079492 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.079636 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.079796 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.079886 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.080096 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.080193 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.080395 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.080484 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.080701 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.080789 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.081005 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.081095 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.081334 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.081426 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.081640 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.081730 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.081944 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.082035 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.082246 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.082334 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.082576 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.082666 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.082886 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.083032 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.083198 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.083295 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.083521 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.083624 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.083833 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.083930 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.084140 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.084228 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.084451 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.084551 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.084771 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.084859 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.085091 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.085185 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.085427 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.085535 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.085738 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.085829 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.086066 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.086156 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.086373 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.086469 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.086684 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.086771 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.086991 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.087090 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.087383 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.088035 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.088340 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.088428 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.088672 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.088805 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.088991 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.089123 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.089327 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.089482 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.089648 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.089792 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.089968 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.090118 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.090298 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.090436 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.090629 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.090757 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.090940 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.091075 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.091269 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.091387 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.091605 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.091702 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.091954 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.092044 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.092267 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.092363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.092612 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.092714 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.092921 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.093011 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.093285 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.093387 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.093568 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.093667 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.093874 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.093962 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.094119 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 25 00:46:57.094209 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.094484 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.094799 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.095125 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.095453 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.095762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.096088 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.096415 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.096760 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.097073 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.097446 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.097775 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.098106 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.098403 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.098710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:46:57.098953 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=105410) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 25 00:46:57.249137 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO dbcounter [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Registered counter for database nova_api Sep 25 00:46:57.259680 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO dbcounter [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Registered counter for database nova_api Sep 25 00:46:57.269420 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_db.sqlalchemy.engines [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105411) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:46:57.276738 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writer thread running {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:46:57.280352 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_db.sqlalchemy.engines [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105410) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:46:57.286881 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writer thread running {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:46:57.334101 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.context [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ade94c2f-42bc-4402-b847-004397ecdfd4(cell1) {{(pid=105411) load_cells /opt/stack/nova/nova/context.py:464}} Sep 25 00:46:57.338499 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_concurrency.lockutils [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105411) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:46:57.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_concurrency.lockutils [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105411) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:46:57.340531 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_concurrency.lockutils [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105411) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:46:57.341283 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_concurrency.lockutils [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105411) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:46:57.341732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_concurrency.lockutils [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105411) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:46:57.343718 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_concurrency.lockutils [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105411) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:46:57.344979 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.context [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ade94c2f-42bc-4402-b847-004397ecdfd4(cell1) {{(pid=105410) load_cells /opt/stack/nova/nova/context.py:464}} Sep 25 00:46:57.345717 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO dbcounter [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Registered counter for database nova_cell0 Sep 25 00:46:57.348915 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:46:57.349414 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:46:57.349746 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO dbcounter [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Registered counter for database nova_cell1 Sep 25 00:46:57.350972 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:46:57.351780 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:46:57.352231 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:46:57.355395 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:46:57.356961 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_db.sqlalchemy.engines [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105411) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:46:57.357682 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO dbcounter [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Registered counter for database nova_cell0 Sep 25 00:46:57.358679 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_db.sqlalchemy.engines [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105411) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:46:57.362044 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO dbcounter [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Registered counter for database nova_cell1 Sep 25 00:46:57.364979 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writer thread running {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:46:57.367284 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writer thread running {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:46:57.369663 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_db.sqlalchemy.engines [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105410) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:46:57.371551 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_db.sqlalchemy.engines [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105410) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:46:57.377599 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writer thread running {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:46:57.379862 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writer thread running {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:46:57.518610 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO dbcounter [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Registered counter for database nova_cell0 Sep 25 00:46:57.530330 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG oslo_db.sqlalchemy.engines [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105411) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:46:57.535579 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writer thread running {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:46:57.540129 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO dbcounter [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Registered counter for database nova_cell0 Sep 25 00:46:57.548538 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_db.sqlalchemy.engines [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=105410) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:46:57.554601 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writer thread running {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:46:57.884992 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: WARNING keystonemiddleware.auth_token [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 25 00:46:57.889206 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: WARNING keystonemiddleware.auth_token [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 25 00:46:57.892307 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: WARNING keystonemiddleware.auth_token [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 25 00:46:57.896446 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: WARNING keystonemiddleware.auth_token [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 25 00:46:58.238001 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: WARNING keystonemiddleware.auth_token [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 25 00:46:58.240147 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: WARNING keystonemiddleware.auth_token [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 25 00:46:58.244526 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: WARNING keystonemiddleware.auth_token [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 25 00:46:58.246612 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: WARNING keystonemiddleware.auth_token [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 25 00:46:58.249841 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55787c5c89c0 pid: 105411 (default app) Sep 25 00:46:58.251823 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55787c5c89c0 pid: 105410 (default app) Sep 25 00:46:58.251916 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:46:58.253113 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-27fef2b5-fbe3-4eb8-8813-d11588bf7b57 None None] 10.222.0.52 "GET /compute/v2.1/" status: 200 len: 388 microversion: 2.1 time: 0.001745 Sep 25 00:46:58.253557 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 788 bytes} [Sun Sep 25 00:46:58 2022] GET /compute/v2.1/ => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:22.984223 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:22.985752 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002281 Sep 25 00:48:22.986161 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:22 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:23.004187 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writing DB stats nova_api:SELECT=3 {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:23.005236 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writing DB stats nova_cell0:SELECT=3 {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:23.006109 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writing DB stats nova_cell1:SELECT=3 {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:23.006874 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:24.335324 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-03e1af45-9cce-4d1d-8baf-4bc9f64de807 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:24.374482 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-03e1af45-9cce-4d1d-8baf-4bc9f64de807 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 15 microversion: 2.61 time: 1.380575 Sep 25 00:48:24.375429 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 2/3] 10.222.0.52 () {60 vars in 1292 bytes} [Sun Sep 25 00:48:22 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 15 bytes in 1382 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:48:27.331740 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:27.333742 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-9dfc3326-4d89-485c-a0f8-786b91fd09f5 None None] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002275 Sep 25 00:48:27.334227 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 2/4] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:27 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:28.026821 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-2cb43128-2cfc-4631-ab39-39456098cf16 admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 256, "os-flavor-access:is_public": true, "id": "c1", "rxtx_factor": 1.0, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 1, "name": "cirros256", "disk": 1, "swap": 0}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:28.093351 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-2cb43128-2cfc-4631-ab39-39456098cf16 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.61 time: 0.751176 Sep 25 00:48:28.094841 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 3/5] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:48:27 2022] POST /compute/v2.1/flavors => generated 407 bytes in 754 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:48:28.119425 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_api:SELECT=3 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:28.120789 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_cell0:SELECT=3 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:28.124325 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_cell1:SELECT=3 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:28.125522 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_cell0:SELECT=6,nova_cell0:INSERT=1 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:28.354559 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-05ac829d-69c7-4a61-a342-dc92a7268c3a admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:28.436656 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-05ac829d-69c7-4a61-a342-dc92a7268c3a admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/c1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.328448 Sep 25 00:48:28.437311 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 3/6] 10.222.0.52 () {64 vars in 1336 bytes} [Sun Sep 25 00:48:28 2022] POST /compute/v2.1/flavors/c1/os-extra_specs => generated 43 bytes in 330 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:48:31.809035 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-2cb43128-2cfc-4631-ab39-39456098cf16 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:31.810196 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-2cb43128-2cfc-4631-ab39-39456098cf16 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001503 Sep 25 00:48:31.810673 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 4/7] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:31 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:32.715907 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-e433877f-66ec-4f60-99bd-3210baf60178 admin admin] Action: 'create', calling method: >, body: {"flavor": {"swap": 0, "ram": 512, "id": "d1", "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "vcpus": 1, "disk": 5, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds512M"}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:32.767917 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-e433877f-66ec-4f60-99bd-3210baf60178 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.949981 Sep 25 00:48:32.768606 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 4/8] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:48:31 2022] POST /compute/v2.1/flavors => generated 404 bytes in 952 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:48:32.786888 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-593a3017-0aa5-4dfd-8915-4670a1ab0a8f admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:32.878911 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-593a3017-0aa5-4dfd-8915-4670a1ab0a8f admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/d1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.099324 Sep 25 00:48:32.880054 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 5/9] 10.222.0.52 () {64 vars in 1336 bytes} [Sun Sep 25 00:48:32 2022] POST /compute/v2.1/flavors/d1/os-extra_specs => generated 43 bytes in 102 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:48:35.880686 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-e433877f-66ec-4f60-99bd-3210baf60178 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:35.882734 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-e433877f-66ec-4f60-99bd-3210baf60178 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002264 Sep 25 00:48:35.883149 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:35 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:36.484194 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-11b32917-8547-49f3-82f9-ce0565403f19 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 10, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "swap": 0, "vcpus": 1, "name": "ds1G", "id": "d2", "ram": 1024}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:36.520091 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-11b32917-8547-49f3-82f9-ce0565403f19 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.629918 Sep 25 00:48:36.521309 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 6/11] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:48:35 2022] POST /compute/v2.1/flavors => generated 404 bytes in 632 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:48:36.534069 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-64ec5150-a115-45d2-900f-3556b24a774d admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:36.567104 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-64ec5150-a115-45d2-900f-3556b24a774d admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/d2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.036770 Sep 25 00:48:36.567825 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 6/12] 10.222.0.52 () {64 vars in 1336 bytes} [Sun Sep 25 00:48:36 2022] POST /compute/v2.1/flavors/d2/os-extra_specs => generated 43 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:48:39.453820 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-11b32917-8547-49f3-82f9-ce0565403f19 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:39.455020 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-11b32917-8547-49f3-82f9-ce0565403f19 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001488 Sep 25 00:48:39.455512 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 7/13] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:39 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:39.996223 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-917bad6c-90af-4220-a9da-81696e5d6434 admin admin] Action: 'create', calling method: >, body: {"flavor": {"id": "d3", "swap": 0, "ram": 2048, "disk": 10, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "vcpus": 2, "rxtx_factor": 1.0, "name": "ds2G"}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:40.030328 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-917bad6c-90af-4220-a9da-81696e5d6434 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.566695 Sep 25 00:48:40.030973 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 7/14] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:48:39 2022] POST /compute/v2.1/flavors => generated 404 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 00:48:40.045160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-e6ba9944-17fa-4f8a-a2a7-bf350b24e0d2 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:40.077475 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-e6ba9944-17fa-4f8a-a2a7-bf350b24e0d2 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/d3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.036341 Sep 25 00:48:40.077992 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 8/15] 10.222.0.52 () {64 vars in 1336 bytes} [Sun Sep 25 00:48:40 2022] POST /compute/v2.1/flavors/d3/os-extra_specs => generated 43 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:48:43.134620 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-917bad6c-90af-4220-a9da-81696e5d6434 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:43.135888 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-917bad6c-90af-4220-a9da-81696e5d6434 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001978 Sep 25 00:48:43.136331 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 8/16] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:43 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:43.677452 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-b98ed728-2821-4f19-8fc8-358530f6d278 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "swap": 0, "ram": 4096, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "ds4G", "id": "d4", "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "disk": 20}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:43.701893 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-b98ed728-2821-4f19-8fc8-358530f6d278 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.61 time: 0.558724 Sep 25 00:48:43.702491 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 9/17] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:48:43 2022] POST /compute/v2.1/flavors => generated 404 bytes in 560 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 00:48:43.715291 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-24cff9cf-dd93-4464-a3f7-9489777b40de admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:43.744573 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-24cff9cf-dd93-4464-a3f7-9489777b40de admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/d4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.032748 Sep 25 00:48:43.745350 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 9/18] 10.222.0.52 () {64 vars in 1336 bytes} [Sun Sep 25 00:48:43 2022] POST /compute/v2.1/flavors/d4/os-extra_specs => generated 43 bytes in 34 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:48:46.681380 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-b98ed728-2821-4f19-8fc8-358530f6d278 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:46.682464 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-b98ed728-2821-4f19-8fc8-358530f6d278 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001539 Sep 25 00:48:46.682739 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 10/19] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:46 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:48:47.299063 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-a2251d61-9237-4b92-9860-a990518f81fd admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:47.333907 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-a2251d61-9237-4b92-9860-a990518f81fd admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 2106 microversion: 2.61 time: 0.642987 Sep 25 00:48:47.333907 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 10/20] 10.222.0.52 () {60 vars in 1292 bytes} [Sun Sep 25 00:48:46 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 2106 bytes in 645 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:48:50.189912 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-b98ed728-2821-4f19-8fc8-358530f6d278 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:50.191097 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-b98ed728-2821-4f19-8fc8-358530f6d278 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001947 Sep 25 00:48:50.191704 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 11/21] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:50 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:50.735971 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-9a50bb46-dac5-4c6a-9a89-80a9785bf7ff admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "rxtx_factor": 1.0, "id": "1", "disk": 1, "OS-FLV-EXT-DATA:ephemeral": 0, "name": "m1.tiny", "vcpus": 1, "ram": 512, "swap": 0}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:50.770355 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-9a50bb46-dac5-4c6a-9a89-80a9785bf7ff admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 402 microversion: 2.61 time: 0.570389 Sep 25 00:48:50.770943 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 11/22] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:48:50 2022] POST /compute/v2.1/flavors => generated 402 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:48:50.790197 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-29063463-7a1e-4956-88e2-c7f0c3fd9d01 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:50.825753 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-29063463-7a1e-4956-88e2-c7f0c3fd9d01 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/1/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.041704 Sep 25 00:48:50.826694 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 12/23] 10.222.0.52 () {64 vars in 1333 bytes} [Sun Sep 25 00:48:50 2022] POST /compute/v2.1/flavors/1/os-extra_specs => generated 43 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:48:53.749212 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-9a50bb46-dac5-4c6a-9a89-80a9785bf7ff admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:53.750379 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-9a50bb46-dac5-4c6a-9a89-80a9785bf7ff admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001610 Sep 25 00:48:53.750807 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 12/24] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:53 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:54.265693 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-6cfa53e6-00a8-47ee-ad42-c3b41353a41c admin admin] Action: 'create', calling method: >, body: {"flavor": {"ram": 2048, "name": "m1.small", "vcpus": 1, "disk": 20, "OS-FLV-EXT-DATA:ephemeral": 0, "os-flavor-access:is_public": true, "rxtx_factor": 1.0, "id": "2", "swap": 0}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:54.273073 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writing DB stats nova_api:SELECT=30,nova_api:INSERT=6,nova_api:SAVEPOINT=3,nova_api:RELEASE=3 {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:48:54.288707 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-6cfa53e6-00a8-47ee-ad42-c3b41353a41c admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.528648 Sep 25 00:48:54.289307 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 13/25] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:48:53 2022] POST /compute/v2.1/flavors => generated 405 bytes in 530 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 00:48:54.301229 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-a9e82c77-7d60-4600-b493-b6f8db1db961 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:54.331241 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-a9e82c77-7d60-4600-b493-b6f8db1db961 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/2/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.033616 Sep 25 00:48:54.333345 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 13/26] 10.222.0.52 () {64 vars in 1333 bytes} [Sun Sep 25 00:48:54 2022] POST /compute/v2.1/flavors/2/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:48:57.013858 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-6cfa53e6-00a8-47ee-ad42-c3b41353a41c admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:48:57.015609 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-6cfa53e6-00a8-47ee-ad42-c3b41353a41c admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002215 Sep 25 00:48:57.016021 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 14/27] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:48:57 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:48:57.498644 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-8f30f097-cb78-4d7f-82e7-aafb0da4a6fd admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.medium", "disk": 40, "id": "3", "swap": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true, "ram": 4096, "vcpus": 2, "OS-FLV-EXT-DATA:ephemeral": 0}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:57.522705 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-8f30f097-cb78-4d7f-82e7-aafb0da4a6fd admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 406 microversion: 2.61 time: 0.499280 Sep 25 00:48:57.523269 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 14/28] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:48:57 2022] POST /compute/v2.1/flavors => generated 406 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 00:48:57.536192 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-86bfeb37-14af-437f-bf56-5977b1e0e930 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:48:57.573683 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-86bfeb37-14af-437f-bf56-5977b1e0e930 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/3/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.041165 Sep 25 00:48:57.574379 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 15/29] 10.222.0.52 () {64 vars in 1333 bytes} [Sun Sep 25 00:48:57 2022] POST /compute/v2.1/flavors/3/os-extra_specs => generated 43 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:49:00.260870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-8f30f097-cb78-4d7f-82e7-aafb0da4a6fd admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:00.261938 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-8f30f097-cb78-4d7f-82e7-aafb0da4a6fd admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001509 Sep 25 00:49:00.262585 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 15/30] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:49:00 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:00.720448 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-9f8c3e9e-85e1-497f-8d86-98c7cab1d8b7 admin admin] Action: 'create', calling method: >, body: {"flavor": {"vcpus": 4, "swap": 0, "os-flavor-access:is_public": true, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "disk": 80, "id": "4", "ram": 8192, "name": "m1.large"}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:49:00.745636 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-9f8c3e9e-85e1-497f-8d86-98c7cab1d8b7 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.61 time: 0.476835 Sep 25 00:49:00.746380 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 16/31] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:49:00 2022] POST /compute/v2.1/flavors => generated 405 bytes in 478 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:49:00.760825 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-d1b2f233-892f-4bc0-95b5-f120e9e38534 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:49:00.766870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_api:SELECT=39,nova_api:SAVEPOINT=4,nova_api:INSERT=8,nova_api:RELEASE=4 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:00.795121 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-d1b2f233-892f-4bc0-95b5-f120e9e38534 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/4/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.039583 Sep 25 00:49:00.796313 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 16/32] 10.222.0.52 () {64 vars in 1333 bytes} [Sun Sep 25 00:49:00 2022] POST /compute/v2.1/flavors/4/os-extra_specs => generated 43 bytes in 42 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:49:03.558718 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-9f8c3e9e-85e1-497f-8d86-98c7cab1d8b7 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:03.559813 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-9f8c3e9e-85e1-497f-8d86-98c7cab1d8b7 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001538 Sep 25 00:49:03.560365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 17/33] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:49:03 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:04.081024 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-55399d7b-7340-4f9b-ab1d-76be9231def3 admin admin] Action: 'create', calling method: >, body: {"flavor": {"os-flavor-access:is_public": true, "disk": 160, "ram": 16384, "name": "m1.xlarge", "rxtx_factor": 1.0, "id": "5", "vcpus": 8, "OS-FLV-EXT-DATA:ephemeral": 0, "swap": 0}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:49:04.102095 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-55399d7b-7340-4f9b-ab1d-76be9231def3 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 408 microversion: 2.61 time: 0.535547 Sep 25 00:49:04.102843 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 17/34] 10.222.0.52 () {64 vars in 1283 bytes} [Sun Sep 25 00:49:03 2022] POST /compute/v2.1/flavors => generated 408 bytes in 537 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 00:49:04.115662 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-826b6008-aa0a-48f9-85d9-a5b6c9d63315 admin admin] Action: 'create', calling method: >, body: {"extra_specs": {"hw_rng:allowed": "True"}} {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:49:04.142126 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-826b6008-aa0a-48f9-85d9-a5b6c9d63315 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors/5/os-extra_specs" status: 200 len: 43 microversion: 2.61 time: 0.030901 Sep 25 00:49:04.145398 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 18/35] 10.222.0.52 () {64 vars in 1333 bytes} [Sun Sep 25 00:49:04 2022] POST /compute/v2.1/flavors/5/os-extra_specs => generated 43 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:49:20.480533 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-55399d7b-7340-4f9b-ab1d-76be9231def3 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:20.482303 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-55399d7b-7340-4f9b-ab1d-76be9231def3 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002407 Sep 25 00:49:20.483004 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 18/36] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:49:20 2022] GET /compute/v2.1 => generated 388 bytes in 4 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:20.496525 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writing DB stats nova_api:INSERT=4,nova_api:SELECT=17,nova_api:SAVEPOINT=2,nova_api:RELEASE=2 {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:20.953724 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-9d8f112f-b43a-4880-be5d-ac6880bfde25 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:20.974393 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-9d8f112f-b43a-4880-be5d-ac6880bfde25 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 4202 microversion: 2.61 time: 0.482591 Sep 25 00:49:20.974965 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 19/37] 10.222.0.52 () {60 vars in 1292 bytes} [Sun Sep 25 00:49:20 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 4202 bytes in 485 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:49:22.554527 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-55399d7b-7340-4f9b-ab1d-76be9231def3 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:22.555488 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-55399d7b-7340-4f9b-ab1d-76be9231def3 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001595 Sep 25 00:49:22.555926 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 19/38] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:49:22 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:23.170386 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-07e0902f-5f6c-416a-9819-8b98d145075d admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:23.176688 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.wsgi [None req-07e0902f-5f6c-416a-9819-8b98d145075d admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-07e0902f-5f6c-416a-9819-8b98d145075d admin admin] Request method failure captured: Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Accept: application/json Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Accept-Encoding: gzip, deflate Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Connection: keep-alive Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Content-Length: 0 Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Host: 10.222.0.52 Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Openstack-Api-Version: compute 2.92 Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Openstack-System-Scope: None Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: User-Agent: python-novaclient Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Auth-Token: gAAAAABjL6WSII1NUdQKo6riH1YuT-OD42uB2f4pcqWVEtKLBUIleIgErklHHNGn-exgNtJvaKB7MNpMlO5cLsiVwj1aXiXFZkKpdVY75QeK7SeB4fNe_U1fO727_--QFN66eIpKuct24hccGxtc5JspfngJuPY7xR4QcdjStAmQoePx70gBNpw Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Domain-Id: None Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Domain-Name: None Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Identity-Status: Confirmed Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Is-Admin-Project: True Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Openstack-Nova-Api-Version: 2.92 Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Domain-Id: default Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Domain-Name: Default Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Name: admin Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Role: member,admin,reader Sep 25 00:49:23.184496 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Roles: member,admin,reader Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c8e32a9668254c2e96354fd797883364", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c8e32a9668254c2e96354fd797883364"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}] Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant: admin Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant-Name: admin Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User: admin Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Domain-Id: default Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Domain-Name: Default Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Id: d6d73e23626043529ac5b9e3889933d4 Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Name: admin Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: method: > Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: exception: Flavor 42 could not be found. Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: action_args: {'id': '42'} Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:49:23.186365 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:23.187392 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.188818 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-07e0902f-5f6c-416a-9819-8b98d145075d admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105411) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:49:23.190333 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-07e0902f-5f6c-416a-9819-8b98d145075d admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.484430 Sep 25 00:49:23.190333 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 20/39] 10.222.0.52 () {60 vars in 1190 bytes} [Sun Sep 25 00:49:22 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 486 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 25 00:49:23.200414 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-70bcd442-8640-48d4-b73a-04cb0f6f86cf admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:23.205050 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_api:SELECT=8,nova_api:SAVEPOINT=1,nova_api:INSERT=2,nova_api:RELEASE=1 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:49:23.227554 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-70bcd442-8640-48d4-b73a-04cb0f6f86cf admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.030884 Sep 25 00:49:23.228115 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 20/40] 10.222.0.52 () {60 vars in 1210 bytes} [Sun Sep 25 00:49:23 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:49:23.240704 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-b56b485c-a52a-4942-b6e1-a6d774373e07 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:23.265809 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-b56b485c-a52a-4942-b6e1-a6d774373e07 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.030697 Sep 25 00:49:23.266520 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 21/41] 10.222.0.52 () {60 vars in 1210 bytes} [Sun Sep 25 00:49:23 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:49:23.278056 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-cfa3fce7-c98a-4f2e-a349-6299b85ec80b admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:23.284088 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.wsgi [None req-cfa3fce7-c98a-4f2e-a349-6299b85ec80b admin admin] HTTP exception thrown: Flavor 42 could not be found. Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-cfa3fce7-c98a-4f2e-a349-6299b85ec80b admin admin] Request method failure captured: Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: request: GET /compute/v2.1/flavors/42 HTTP/1.1 Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Accept: application/json Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Accept-Encoding: gzip, deflate Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Connection: keep-alive Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Content-Length: 0 Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Host: 10.222.0.52 Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Openstack-Api-Version: compute 2.92 Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Openstack-System-Scope: None Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: User-Agent: python-novaclient Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Auth-Token: gAAAAABjL6WSII1NUdQKo6riH1YuT-OD42uB2f4pcqWVEtKLBUIleIgErklHHNGn-exgNtJvaKB7MNpMlO5cLsiVwj1aXiXFZkKpdVY75QeK7SeB4fNe_U1fO727_--QFN66eIpKuct24hccGxtc5JspfngJuPY7xR4QcdjStAmQoePx70gBNpw Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Domain-Id: None Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Domain-Name: None Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Identity-Status: Confirmed Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Is-Admin-Project: True Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Openstack-Nova-Api-Version: 2.92 Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Domain-Id: default Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Domain-Name: Default Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Name: admin Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Role: member,admin,reader Sep 25 00:49:23.288411 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Roles: member,admin,reader Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c8e32a9668254c2e96354fd797883364", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c8e32a9668254c2e96354fd797883364"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}] Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant: admin Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant-Name: admin Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User: admin Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Domain-Id: default Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Domain-Name: Default Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Id: d6d73e23626043529ac5b9e3889933d4 Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Name: admin Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: method: > Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: exception: Flavor 42 could not be found. Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: action_args: {'id': '42'} Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:49:23.290595 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 42 could not be found. Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:23.291710 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 42 could not be found. Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 42 could not be found. Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:23.293094 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-cfa3fce7-c98a-4f2e-a349-6299b85ec80b admin admin] Returning 404 to user: Flavor 42 could not be found. {{(pid=105410) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:49:23.294475 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-cfa3fce7-c98a-4f2e-a349-6299b85ec80b admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/42" status: 404 len: 75 microversion: 2.92 time: 0.015903 Sep 25 00:49:23.294475 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 21/42] 10.222.0.52 () {60 vars in 1190 bytes} [Sun Sep 25 00:49:23 2022] GET /compute/v2.1/flavors/42 => generated 75 bytes in 17 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 25 00:49:24.724372 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-b56b485c-a52a-4942-b6e1-a6d774373e07 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:24.725543 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-b56b485c-a52a-4942-b6e1-a6d774373e07 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001791 Sep 25 00:49:24.725988 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 22/43] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:49:24 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:25.328762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-554be2e6-56fe-4cfb-b306-e35f1fcfde53 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:25.334794 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.wsgi [None req-554be2e6-56fe-4cfb-b306-e35f1fcfde53 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-554be2e6-56fe-4cfb-b306-e35f1fcfde53 admin admin] Request method failure captured: Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Accept: application/json Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Accept-Encoding: gzip, deflate Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Connection: keep-alive Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Content-Length: 0 Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Host: 10.222.0.52 Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Openstack-Api-Version: compute 2.92 Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Openstack-System-Scope: None Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: User-Agent: python-novaclient Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Auth-Token: gAAAAABjL6WUBEC0P19F4O7QPXOflLXFM3HR-3u8oHVQpYhvqQ-hKfsKI2z7NLccs88JI-_hcd9K4SA3tTeg0ruB28J9M-_FXf5WEbwx7ATFyC5GC90U5NXzDJl8qOPi8yTISsWbPNNeJe1-8GxH0o3698OjzC61GvL_OB-2j2NsmIbKbSkxcF8 Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Domain-Id: None Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Domain-Name: None Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Identity-Status: Confirmed Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Is-Admin-Project: True Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Openstack-Nova-Api-Version: 2.92 Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Domain-Id: default Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Domain-Name: Default Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Name: admin Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Role: member,admin,reader Sep 25 00:49:25.336363 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Roles: member,admin,reader Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c8e32a9668254c2e96354fd797883364", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c8e32a9668254c2e96354fd797883364"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}] Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant: admin Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant-Name: admin Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User: admin Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Domain-Id: default Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Domain-Name: Default Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Id: d6d73e23626043529ac5b9e3889933d4 Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Name: admin Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: method: > Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: exception: Flavor 84 could not be found. Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: action_args: {'id': '84'} Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:49:25.338977 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:25.340762 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.343186 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-554be2e6-56fe-4cfb-b306-e35f1fcfde53 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105410) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:49:25.345441 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-554be2e6-56fe-4cfb-b306-e35f1fcfde53 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.462732 Sep 25 00:49:25.345441 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 22/44] 10.222.0.52 () {60 vars in 1190 bytes} [Sun Sep 25 00:49:24 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 464 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 25 00:49:25.351561 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-fc08a23d-7df0-42f4-a31b-671cbbc1330a admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:25.367765 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-fc08a23d-7df0-42f4-a31b-671cbbc1330a admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.020620 Sep 25 00:49:25.368353 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 23/45] 10.222.0.52 () {60 vars in 1210 bytes} [Sun Sep 25 00:49:25 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:49:25.378799 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-1298c45b-d032-45c3-b2d7-aed65fcdf8a6 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:25.398197 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-1298c45b-d032-45c3-b2d7-aed65fcdf8a6 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.023026 Sep 25 00:49:25.398761 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 23/46] 10.222.0.52 () {60 vars in 1210 bytes} [Sun Sep 25 00:49:25 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 25 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:49:25.412932 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-e89175a5-179a-4e9c-a14f-754527d9bb66 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:25.418269 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.wsgi [None req-e89175a5-179a-4e9c-a14f-754527d9bb66 admin admin] HTTP exception thrown: Flavor 84 could not be found. Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-e89175a5-179a-4e9c-a14f-754527d9bb66 admin admin] Request method failure captured: Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: request: GET /compute/v2.1/flavors/84 HTTP/1.1 Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Accept: application/json Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Accept-Encoding: gzip, deflate Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Connection: keep-alive Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Content-Length: 0 Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Host: 10.222.0.52 Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Openstack-Api-Version: compute 2.92 Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Openstack-System-Scope: None Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: User-Agent: python-novaclient Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Auth-Token: gAAAAABjL6WUBEC0P19F4O7QPXOflLXFM3HR-3u8oHVQpYhvqQ-hKfsKI2z7NLccs88JI-_hcd9K4SA3tTeg0ruB28J9M-_FXf5WEbwx7ATFyC5GC90U5NXzDJl8qOPi8yTISsWbPNNeJe1-8GxH0o3698OjzC61GvL_OB-2j2NsmIbKbSkxcF8 Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Domain-Id: None Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Domain-Name: None Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Identity-Status: Confirmed Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Is-Admin-Project: True Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Openstack-Nova-Api-Version: 2.92 Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Domain-Id: default Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Domain-Name: Default Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Name: admin Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Role: member,admin,reader Sep 25 00:49:25.419824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Roles: member,admin,reader Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c8e32a9668254c2e96354fd797883364", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c8e32a9668254c2e96354fd797883364"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}] Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant: admin Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant-Name: admin Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User: admin Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Domain-Id: default Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Domain-Name: Default Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Id: d6d73e23626043529ac5b9e3889933d4 Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Name: admin Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: method: > Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: exception: Flavor 84 could not be found. Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: action_args: {'id': '84'} Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:49:25.422994 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 84 could not be found. Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:25.423987 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 84 could not be found. Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 84 could not be found. Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:25.425357 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-e89175a5-179a-4e9c-a14f-754527d9bb66 admin admin] Returning 404 to user: Flavor 84 could not be found. {{(pid=105411) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:49:25.426622 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-e89175a5-179a-4e9c-a14f-754527d9bb66 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/84" status: 404 len: 75 microversion: 2.92 time: 0.011976 Sep 25 00:49:25.426622 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 24/47] 10.222.0.52 () {60 vars in 1190 bytes} [Sun Sep 25 00:49:25 2022] GET /compute/v2.1/flavors/84 => generated 75 bytes in 13 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 25 00:49:26.787564 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-1298c45b-d032-45c3-b2d7-aed65fcdf8a6 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:26.789321 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-1298c45b-d032-45c3-b2d7-aed65fcdf8a6 admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002352 Sep 25 00:49:26.789730 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 24/48] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:49:26 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:49:27.490940 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-3edaf2c4-e8b7-4c68-8d60-4701d2c46adc admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:27.497105 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.wsgi [None req-3edaf2c4-e8b7-4c68-8d60-4701d2c46adc admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-3edaf2c4-e8b7-4c68-8d60-4701d2c46adc admin admin] Request method failure captured: Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Accept: application/json Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Accept-Encoding: gzip, deflate Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Connection: keep-alive Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Content-Length: 0 Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Host: 10.222.0.52 Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Openstack-Api-Version: compute 2.92 Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Openstack-System-Scope: None Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: User-Agent: python-novaclient Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Auth-Token: gAAAAABjL6WWDJJ7tjNUrT3F9kIcuj2v07ztA1iYuehDg9R8WOPLs2wHEozDY0_3m9oyuarfBB5CXwv-QG-XLF2I-6AWphxrk4jhmQCyB1poKzEw9lk1L_qnBviqLF7Z54jwnNyvyAEWgrXXbXjr_firWZ3VMMnyEoa1E7nafd3K0pE9DsjKYxo Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Domain-Id: None Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Domain-Name: None Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Identity-Status: Confirmed Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Is-Admin-Project: True Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Openstack-Nova-Api-Version: 2.92 Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Domain-Id: default Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Domain-Name: Default Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Project-Name: admin Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Role: member,admin,reader Sep 25 00:49:27.498732 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Roles: member,admin,reader Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c8e32a9668254c2e96354fd797883364", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c8e32a9668254c2e96354fd797883364"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}] Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant: admin Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-Tenant-Name: admin Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User: admin Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Domain-Id: default Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Domain-Name: Default Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Id: d6d73e23626043529ac5b9e3889933d4 Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: X-User-Name: admin Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: method: > Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: exception: Flavor 451 could not be found. Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: action_args: {'id': '451'} Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:49:27.501229 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:27.503160 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.505562 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-3edaf2c4-e8b7-4c68-8d60-4701d2c46adc admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105411) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:49:27.508005 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-3edaf2c4-e8b7-4c68-8d60-4701d2c46adc admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.560762 Sep 25 00:49:27.508005 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 25/49] 10.222.0.52 () {60 vars in 1193 bytes} [Sun Sep 25 00:49:26 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 562 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 25 00:49:27.514136 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-61611628-96c8-413d-80be-98e3a185d899 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:27.531867 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-61611628-96c8-413d-80be-98e3a185d899 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.021259 Sep 25 00:49:27.532440 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 25/50] 10.222.0.52 () {60 vars in 1210 bytes} [Sun Sep 25 00:49:27 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:49:27.546255 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:27.563079 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] 10.222.0.52 "GET /compute/v2.1/flavors?is_public=None" status: 200 len: 2056 microversion: 2.92 time: 0.020348 Sep 25 00:49:27.563665 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 26/51] 10.222.0.52 () {60 vars in 1210 bytes} [Sun Sep 25 00:49:27 2022] GET /compute/v2.1/flavors?is_public=None => generated 2056 bytes in 22 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:49:27.574776 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-ad967d28-f843-48e2-9393-e71a18b07971 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:27.580372 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.wsgi [None req-ad967d28-f843-48e2-9393-e71a18b07971 admin admin] HTTP exception thrown: Flavor 451 could not be found. Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-ad967d28-f843-48e2-9393-e71a18b07971 admin admin] Request method failure captured: Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: request: GET /compute/v2.1/flavors/451 HTTP/1.1 Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Accept: application/json Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Accept-Encoding: gzip, deflate Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Connection: keep-alive Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Content-Length: 0 Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Host: 10.222.0.52 Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Openstack-Api-Version: compute 2.92 Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Openstack-System-Scope: None Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: User-Agent: python-novaclient Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Auth-Token: gAAAAABjL6WWDJJ7tjNUrT3F9kIcuj2v07ztA1iYuehDg9R8WOPLs2wHEozDY0_3m9oyuarfBB5CXwv-QG-XLF2I-6AWphxrk4jhmQCyB1poKzEw9lk1L_qnBviqLF7Z54jwnNyvyAEWgrXXbXjr_firWZ3VMMnyEoa1E7nafd3K0pE9DsjKYxo Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Domain-Id: None Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Domain-Name: None Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Identity-Status: Confirmed Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Is-Admin-Project: True Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Openstack-Nova-Api-Version: 2.92 Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Domain-Id: default Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Domain-Name: Default Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Project-Name: admin Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Role: member,admin,reader Sep 25 00:49:27.582037 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Roles: member,admin,reader Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_c8e32a9668254c2e96354fd797883364", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/c8e32a9668254c2e96354fd797883364"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/c8e32a9668254c2e96354fd797883364"}]}] Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant: admin Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant-Id: c8e32a9668254c2e96354fd797883364 Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-Tenant-Name: admin Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User: admin Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Domain-Id: default Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Domain-Name: Default Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Id: d6d73e23626043529ac5b9e3889933d4 Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: X-User-Name: admin Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: method: > Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: exception: Flavor 451 could not be found. Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: action_args: {'id': '451'} Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:49:27.584745 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 66, in show Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi flavor = flavors.get_flavor_by_flavor_id(id, ctxt=context) Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/flavors.py", line 139, in get_flavor_by_flavor_id Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return objects.Flavor.get_by_flavor_id(ctxt, flavorid, read_deleted) Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 390, in get_by_flavor_id Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi db_flavor = cls._flavor_get_by_flavor_id_from_db(context, Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/db/utils.py", line 35, in wrapper Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/flavor.py", line 317, in _flavor_get_by_flavor_id_from_db Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise exception.FlavorNotFound(flavor_id=flavor_id) Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi nova.exception.FlavorNotFound: Flavor 451 could not be found. Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:27.586870 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/flavors.py", line 68, in show Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Flavor 451 could not be found. Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Flavor 451 could not be found. Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: ERROR nova.api.openstack.wsgi Sep 25 00:49:27.589627 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-ad967d28-f843-48e2-9393-e71a18b07971 admin admin] Returning 404 to user: Flavor 451 could not be found. {{(pid=105410) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:49:27.592002 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-ad967d28-f843-48e2-9393-e71a18b07971 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/451" status: 404 len: 76 microversion: 2.92 time: 0.012620 Sep 25 00:49:27.592002 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 26/52] 10.222.0.52 () {60 vars in 1193 bytes} [Sun Sep 25 00:49:27 2022] GET /compute/v2.1/flavors/451 => generated 76 bytes in 15 msecs (HTTP/1.1 404) 9 headers in 380 bytes (1 switches on core 0) Sep 25 00:49:28.985446 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:28.987007 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001691 Sep 25 00:49:28.987476 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 27/53] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:49:28 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:29.617780 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-64868826-a060-49fe-8ce9-0937fe285721 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.nano", "ram": 256, "vcpus": 1, "disk": 1, "id": "42", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:49:29.641939 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-64868826-a060-49fe-8ce9-0937fe285721 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 404 microversion: 2.92 time: 0.494608 Sep 25 00:49:29.642505 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 27/54] 10.222.0.52 () {64 vars in 1235 bytes} [Sun Sep 25 00:49:29 2022] POST /compute/v2.1/flavors => generated 404 bytes in 497 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 00:49:30.947323 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:30.948342 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001441 Sep 25 00:49:30.948738 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 28/55] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:49:30 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:31.484404 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-742f2ebe-b636-41c1-841b-16519989e86b admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.micro", "ram": 256, "vcpus": 1, "disk": 2, "id": "84", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:49:31.504000 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-742f2ebe-b636-41c1-841b-16519989e86b admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 405 microversion: 2.92 time: 0.415940 Sep 25 00:49:31.504542 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 28/56] 10.222.0.52 () {64 vars in 1235 bytes} [Sun Sep 25 00:49:31 2022] POST /compute/v2.1/flavors => generated 405 bytes in 417 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:49:32.752671 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:32.753592 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001783 Sep 25 00:49:32.753988 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 29/57] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:49:32 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:33.299671 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-bdc96dcc-c0c6-453b-a725-f128d3964ab4 admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat", "ram": 512, "vcpus": 1, "disk": 1, "id": "451", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:49:33.320739 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-bdc96dcc-c0c6-453b-a725-f128d3964ab4 admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 407 microversion: 2.92 time: 0.416997 Sep 25 00:49:33.321202 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 29/58] 10.222.0.52 () {64 vars in 1235 bytes} [Sun Sep 25 00:49:32 2022] POST /compute/v2.1/flavors => generated 407 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:49:34.567713 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:34.569090 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001805 Sep 25 00:49:34.569521 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 30/59] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:49:34 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:35.143753 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-fc2fb0b2-6ad4-4ea7-8784-a1e85112e65e admin admin] Action: 'create', calling method: >, body: {"flavor": {"name": "m1.heat2", "ram": 1024, "vcpus": 1, "disk": 1, "id": "452", "swap": 0, "OS-FLV-EXT-DATA:ephemeral": 0, "rxtx_factor": 1.0, "os-flavor-access:is_public": true}} {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:49:35.163307 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-fc2fb0b2-6ad4-4ea7-8784-a1e85112e65e admin admin] 10.222.0.52 "POST /compute/v2.1/flavors" status: 200 len: 409 microversion: 2.92 time: 0.453065 Sep 25 00:49:35.163875 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 30/60] 10.222.0.52 () {64 vars in 1235 bytes} [Sun Sep 25 00:49:34 2022] POST /compute/v2.1/flavors => generated 409 bytes in 454 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:49:36.413795 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:36.414799 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001472 Sep 25 00:49:36.415187 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 31/61] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:49:36 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:49:36.972064 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-ea0a4f05-43d6-4efe-a552-0150aab568e7 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:49:36.993570 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-ea0a4f05-43d6-4efe-a552-0150aab568e7 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail" status: 200 len: 5777 microversion: 2.92 time: 0.433731 Sep 25 00:49:36.994097 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 31/62] 10.222.0.52 () {60 vars in 1202 bytes} [Sun Sep 25 00:49:36 2022] GET /compute/v2.1/flavors/detail => generated 5777 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:50:35.964948 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:50:35.965917 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-a97a5102-6466-485a-a20e-a3d05793813f admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001570 Sep 25 00:50:35.966330 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 32/63] 10.222.0.52 () {54 vars in 931 bytes} [Sun Sep 25 00:50:35 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:50:35.977593 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_api:SELECT=26,nova_api:INSERT=4 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:50:36.395306 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-05fcb1d5-1cc6-4b5a-8246-be11b5e60909 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:50:36.417743 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-05fcb1d5-1cc6-4b5a-8246-be11b5e60909 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/detail?is_public=True" status: 200 len: 5791 microversion: 2.61 time: 0.444806 Sep 25 00:50:36.418674 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 32/64] 10.222.0.52 () {60 vars in 1292 bytes} [Sun Sep 25 00:50:35 2022] GET /compute/v2.1/flavors/detail?is_public=True => generated 5791 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:50:36.439897 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writing DB stats nova_api:SELECT=14 {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:50:36.443984 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-46baf86a-2952-4e37-a5cd-a0492820e1e5 admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:50:36.451347 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-46baf86a-2952-4e37-a5cd-a0492820e1e5 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/42/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.012859 Sep 25 00:50:36.451889 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 33/65] 10.222.0.52 () {60 vars in 1283 bytes} [Sun Sep 25 00:50:36 2022] GET /compute/v2.1/flavors/42/os-extra_specs => generated 19 bytes in 14 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:50:36.464774 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-7d7f4f3d-db11-460e-aa68-fa49d4b5e7c8 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:50:36.472219 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-7d7f4f3d-db11-460e-aa68-fa49d4b5e7c8 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/451/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011093 Sep 25 00:50:36.473570 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 33/66] 10.222.0.52 () {60 vars in 1286 bytes} [Sun Sep 25 00:50:36 2022] GET /compute/v2.1/flavors/451/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:50:36.485812 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:50:36.494201 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/452/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.011993 Sep 25 00:50:36.494741 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 34/67] 10.222.0.52 () {60 vars in 1286 bytes} [Sun Sep 25 00:50:36 2022] GET /compute/v2.1/flavors/452/os-extra_specs => generated 19 bytes in 13 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:50:36.509293 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-496408f9-48ba-403f-83b1-19c466dd84a3 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:50:36.520317 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-496408f9-48ba-403f-83b1-19c466dd84a3 admin admin] 10.222.0.52 "GET /compute/v2.1/flavors/84/os-extra_specs" status: 200 len: 19 microversion: 2.61 time: 0.016195 Sep 25 00:50:36.521061 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 34/68] 10.222.0.52 () {60 vars in 1283 bytes} [Sun Sep 25 00:50:36 2022] GET /compute/v2.1/flavors/84/os-extra_specs => generated 19 bytes in 18 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:53:43.756377 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:53:43.757424 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001459 Sep 25 00:53:43.757824 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 35/69] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:53:43 2022] GET /compute/v2.1 => generated 388 bytes in 2 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:53:43.908438 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_api:SELECT=6 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:44.098542 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-d9f35f2c-aa71-49b7-9266-20f7b0f519e1 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:53:44.103334 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-d9f35f2c-aa71-49b7-9266-20f7b0f519e1 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:53:44.103814 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-d9f35f2c-aa71-49b7-9266-20f7b0f519e1 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:53:44.104246 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-d9f35f2c-aa71-49b7-9266-20f7b0f519e1 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:53:44.104959 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-d9f35f2c-aa71-49b7-9266-20f7b0f519e1 admin admin] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:53:44.105447 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-d9f35f2c-aa71-49b7-9266-20f7b0f519e1 admin admin] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:53:44.105869 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-d9f35f2c-aa71-49b7-9266-20f7b0f519e1 admin admin] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:53:44.138133 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-d9f35f2c-aa71-49b7-9266-20f7b0f519e1 admin admin] 10.222.0.52 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.230863 Sep 25 00:53:44.138862 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 35/70] 10.222.0.52 () {60 vars in 1193 bytes} [Sun Sep 25 00:53:43 2022] GET /compute/v2.1/os-services => generated 791 bytes in 232 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 00:53:45.238813 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:53:45.239815 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.001673 Sep 25 00:53:45.240217 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 36/71] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:53:45 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:53:45.434351 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-43599a34-4855-4583-8ccd-107007bb179d admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:53:45.441656 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-43599a34-4855-4583-8ccd-107007bb179d admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:53:45.442130 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-43599a34-4855-4583-8ccd-107007bb179d admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:53:45.442556 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-43599a34-4855-4583-8ccd-107007bb179d admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:53:45.443259 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-43599a34-4855-4583-8ccd-107007bb179d admin admin] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:53:45.443699 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-43599a34-4855-4583-8ccd-107007bb179d admin admin] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:53:45.444114 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-43599a34-4855-4583-8ccd-107007bb179d admin admin] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:53:45.463742 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-43599a34-4855-4583-8ccd-107007bb179d admin admin] 10.222.0.52 "GET /compute/v2.1/os-services" status: 200 len: 791 microversion: 2.92 time: 0.081367 Sep 25 00:53:45.464300 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 36/72] 10.222.0.52 () {60 vars in 1193 bytes} [Sun Sep 25 00:53:45 2022] GET /compute/v2.1/os-services => generated 791 bytes in 83 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:53:56.998460 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:53:56.999593 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002312 Sep 25 00:53:56.999993 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 37/73] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:53:56 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:53:57.150823 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_cell0:SELECT=4 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:57.151539 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_cell1:SELECT=4 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:57.152140 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG dbcounter [-] [105410] Writing DB stats nova_api:SELECT=8 {{(pid=105410) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:53:57.202048 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-f7699ae7-0bef-4838-b368-de637d17c9d8 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:53:57.205552 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-f7699ae7-0bef-4838-b368-de637d17c9d8 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:53:57.205976 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-f7699ae7-0bef-4838-b368-de637d17c9d8 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:53:57.206405 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-f7699ae7-0bef-4838-b368-de637d17c9d8 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:53:57.207116 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-f7699ae7-0bef-4838-b368-de637d17c9d8 admin admin] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:53:57.207561 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-f7699ae7-0bef-4838-b368-de637d17c9d8 admin admin] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:53:57.207979 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-f7699ae7-0bef-4838-b368-de637d17c9d8 admin admin] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:53:57.233268 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-f7699ae7-0bef-4838-b368-de637d17c9d8 admin admin] 10.222.0.52 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.086826 Sep 25 00:53:57.233836 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 37/74] 10.222.0.52 () {60 vars in 1193 bytes} [Sun Sep 25 00:53:57 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:53:58.324761 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG nova.api.openstack.wsgi [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] Calling method '>' {{(pid=105411) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:53:58.326505 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: INFO nova.api.openstack.requestlog [None req-33fdcfd3-cd4e-42c4-9b28-53b27beecc5a admin admin] 10.222.0.52 "GET /compute/v2.1" status: 200 len: 388 microversion: 2.1 time: 0.002102 Sep 25 00:53:58.326903 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: [pid: 105411|app: 0|req: 38/75] 10.222.0.52 () {56 vars in 1074 bytes} [Sun Sep 25 00:53:58 2022] GET /compute/v2.1 => generated 388 bytes in 3 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:53:58.519924 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG nova.api.openstack.wsgi [None req-050dacf5-7bb9-4d0d-9439-9ff000872612 admin admin] Calling method '>' {{(pid=105410) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:53:58.523489 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-050dacf5-7bb9-4d0d-9439-9ff000872612 admin admin] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:53:58.524005 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-050dacf5-7bb9-4d0d-9439-9ff000872612 admin admin] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:53:58.524440 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-050dacf5-7bb9-4d0d-9439-9ff000872612 admin admin] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:53:58.525171 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-050dacf5-7bb9-4d0d-9439-9ff000872612 admin admin] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:53:58.525648 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-050dacf5-7bb9-4d0d-9439-9ff000872612 admin admin] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:53:58.526067 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: DEBUG oslo_concurrency.lockutils [None req-050dacf5-7bb9-4d0d-9439-9ff000872612 admin admin] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=105410) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:53:58.550068 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: INFO nova.api.openstack.requestlog [None req-050dacf5-7bb9-4d0d-9439-9ff000872612 admin admin] 10.222.0.52 "GET /compute/v2.1/os-services" status: 200 len: 1011 microversion: 2.92 time: 0.083979 Sep 25 00:53:58.550646 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: [pid: 105410|app: 0|req: 38/76] 10.222.0.52 () {60 vars in 1193 bytes} [Sun Sep 25 00:53:58 2022] GET /compute/v2.1/os-services => generated 1011 bytes in 85 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:54:56.433661 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: Sun Sep 25 00:54:56 2022 - graceful shutdown triggered... Sep 25 00:54:56.433534 dv-cinder-master-3-iscsi systemd[1]: Stopping Devstack devstack@n-api.service... Sep 25 00:54:56.434653 dv-cinder-master-3-iscsi devstack@n-api.service[105410]: Gracefully killing worker 1 (pid: 105410)... Sep 25 00:54:56.435327 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: Gracefully killing worker 2 (pid: 105411)... Sep 25 00:54:56.564082 dv-cinder-master-3-iscsi devstack@n-api.service[105411]: DEBUG dbcounter [-] [105411] Writing DB stats nova_api:SELECT=4 {{(pid=105411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:54:57.434555 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: worker 1 buried after 1 seconds Sep 25 00:54:57.434555 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: worker 2 buried after 1 seconds Sep 25 00:54:57.435241 dv-cinder-master-3-iscsi devstack@n-api.service[105409]: goodbye to uWSGI. Sep 25 00:54:57.436172 dv-cinder-master-3-iscsi systemd[1]: devstack@n-api.service: Succeeded. Sep 25 00:54:57.436726 dv-cinder-master-3-iscsi systemd[1]: Stopped Devstack devstack@n-api.service. Sep 25 00:54:57.439542 dv-cinder-master-3-iscsi systemd[1]: Starting Devstack devstack@n-api.service... Sep 25 00:54:57.449221 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: [uWSGI] getting INI configuration from /etc/nova/nova-api-uwsgi.ini Sep 25 00:54:57.451967 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Sep 25 00:54:57 2022] *** Sep 25 00:54:57.452124 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: nodename: dv-cinder-master-3-iscsi Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: machine: x86_64 Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: clock source: unix Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: pcre jit disabled Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: detected number of CPU cores: 4 Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: current working directory: / Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: detected binary path: /usr/bin/uwsgi-core Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: your processes number limit is 47680 Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: your memory page size is 4096 bytes Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: detected max file descriptor number: 2048 Sep 25 00:54:57.452285 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: lock engine: pthread robust mutexes Sep 25 00:54:57.453192 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: thunder lock: enabled Sep 25 00:54:57.453192 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/nova-api-wsgi.socket fd 4 Sep 25 00:54:57.453192 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 25 00:54:57.499477 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: Python main interpreter initialized at 0x55b739c259c0 Sep 25 00:54:57.499477 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: python threads support enabled Sep 25 00:54:57.499477 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: your server socket listen backlog is limited to 100 connections Sep 25 00:54:57.499477 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: your mercy for graceful operations on workers is 90 seconds Sep 25 00:54:57.500007 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: mapped 403077 bytes (393 KB) for 2 cores Sep 25 00:54:57.500007 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: *** Operational MODE: preforking *** Sep 25 00:54:57.500217 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: *** uWSGI is running in multiple interpreter mode *** Sep 25 00:54:57.500217 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: spawned uWSGI master process (pid: 120433) Sep 25 00:54:57.500419 dv-cinder-master-3-iscsi systemd[1]: Started Devstack devstack@n-api.service. Sep 25 00:54:57.501324 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: spawned uWSGI worker 1 (pid: 120450, cores: 1) Sep 25 00:54:57.501459 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: spawned uWSGI worker 2 (pid: 120451, cores: 1) Sep 25 00:54:57.501549 dv-cinder-master-3-iscsi devstack@n-api.service[120433]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 25 00:54:58.177158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 25 00:54:58.182516 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Modules with known eventlet monkey patching issues were imported prior to eventlet monkey patching: urllib3. This warning can usually be ignored if the caller is only importing and not executing nova code. Sep 25 00:54:59.371747 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:54:59.371747 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: from cryptography.utils import int_from_bytes Sep 25 00:54:59.372516 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:54:59.372516 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: from cryptography.utils import int_from_bytes Sep 25 00:54:59.402384 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:54:59.402384 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: from cryptography.utils import int_from_bytes Sep 25 00:54:59.402981 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 25 00:54:59.402981 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: from cryptography.utils import int_from_bytes Sep 25 00:55:01.931398 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 25 00:55:01.931964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 25 00:55:01.932305 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 25 00:55:01.932566 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 25 00:55:01.933576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 25 00:55:01.933576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.933773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.934008 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.934342 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.934691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.935011 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.935402 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.935673 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.936013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.936284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.936608 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.937167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.937539 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.937864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.938234 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.938524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.938884 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.939159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.939474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.939990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.940530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.940768 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.941089 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.942470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.942470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.942470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.942470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.943130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.943130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.943422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.943818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.944148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.944472 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.944787 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.945243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.945589 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.945878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.946207 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.946701 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.947124 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.947358 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.947647 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.948543 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.948887 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.949196 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.949538 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.950000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.950332 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.950660 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.950971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.951373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.951518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.951840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.952165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.952496 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.952754 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.953010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.953361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.953692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.954009 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.954318 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.954636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.954945 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.955259 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.955571 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.955881 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.956190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.956520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.956844 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.957171 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.957508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.957833 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.958253 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.52 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.958510 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.52 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.958821 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.959169 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.959492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.959807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.960129 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.960455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.960774 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.961090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.961429 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.961753 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.962097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.962407 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.962666 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.962973 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.963281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.964349 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.964597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.964770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.965084 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.965411 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.965727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.966035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.966344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.966652 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.966961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.967271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.967578 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.967887 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.968197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.968504 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.968841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.969158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.969498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.969810 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.970120 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.970428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.970758 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.971071 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.971405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.971709 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.972022 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.972332 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.972655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.972967 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.973303 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.973631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.973939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.974250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.974573 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.975628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.975628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.975628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.975926 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.976117 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.976396 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.976704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.977009 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.977340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.977672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.978001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.978319 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.978631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.979072 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.979800 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.980323 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.980808 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.981241 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.981390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.981688 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.982024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.982349 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.982853 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.983328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.983508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.983827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.984146 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.984463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.984787 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.985116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.985484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.985809 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.986131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.986456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.986771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.987094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.987428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.987759 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.988080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.988690 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.989044 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.989392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.989723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.990061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.990375 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.990629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 25 00:55:01.991077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.991175 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] Configuration options gathered from: {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 25 00:55:01.991270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.991356 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.991450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] command line args: [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 25 00:55:01.991639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.991778 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] config files: ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 25 00:55:01.991955 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.992059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ================================================================================ {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 25 00:55:01.992278 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.992418 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] allow_resize_to_same_host = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.992595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.992747 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] arq_binding_timeout = 300 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.992925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.993071 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.993241 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.993439 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] block_device_allocate_retries_interval = 3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.993578 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.993742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cert = self.pem {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.993895 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.994057 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute_driver = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.994214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.994369 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute_monitors = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.994527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.994694 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] config_dir = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.994945 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.995705 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] config_drive_format = iso9660 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.995945 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.996117 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] config_file = ['/etc/nova/api-paste.ini', '/etc/nova/nova.conf'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.996259 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.996437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] config_source = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.996689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.996790 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] console_host = dv-cinder-master-3-iscsi {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.996888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.996987 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] control_exchange = nova {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.997100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.997315 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cpu_allocation_ratio = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.997413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.997632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.997735 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] daemon = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.997947 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.998040 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] debug = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.998256 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.998350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] default_access_ip_network_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.998567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.998669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] default_availability_zone = nova {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.998870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] default_ephemeral_format = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.998969 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.999181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.999437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'glanceclient=WARN', 'oslo.privsep.daemon=INFO'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.999572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.999691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] default_schedule_zone = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:01.999785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:01.999996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] disk_allocation_ratio = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.000125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.000309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] enable_new_services = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.000444 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.000650 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] enabled_apis = ['osapi_compute'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.000787 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.001057 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] enabled_ssl_apis = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.001371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.001482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] flat_injected = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.001570 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.001662 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] force_config_drive = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.001748 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.001899 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] force_raw_images = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.002030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.002230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] graceful_shutdown_timeout = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.002367 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.002540 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] heal_instance_info_cache_interval = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.002646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.002944 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.003055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] host = dv-cinder-master-3-iscsi {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.003256 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.003357 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] initial_cpu_allocation_ratio = 16.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.003565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] initial_disk_allocation_ratio = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.003672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.003878 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] initial_ram_allocation_ratio = 1.5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.003978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.004186 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.004305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] injected_network_template = /opt/stack/nova/nova/virt/interfaces.template {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.004610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.004712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] instance_build_timeout = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.004852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.004951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] instance_delete_interval = 300 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.005134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.005346 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] instance_format = [instance: %(uuid)s] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.005555 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] instance_name_template = instance-%08x {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.005727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.005891 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.006085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.006263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.006416 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] instance_usage_audit_period = month {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.006513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.006606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.006777 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.006919 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] instances_path = /opt/stack/data/nova/instances {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.007093 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] internal_service_availability_zone = internal {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.007186 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.007396 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] key = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.007508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.007704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] live_migration_retry_count = 30 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.007809 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.008018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] log_config_append = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.008118 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.008344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.008447 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.008651 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] log_dir = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.008756 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.008957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] log_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.009052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.009207 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] log_options = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.009355 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.009556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.009690 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.009878 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] log_rotate_interval_type = days {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.009981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.010196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] log_rotation_type = none {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.010290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.010574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.010708 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.010895 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.011070 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.011240 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.011390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.011527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.011617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.011795 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.011912 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.012104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.012223 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] long_rpc_timeout = 1800 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.012922 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.013036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_builds = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.013133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.013229 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_live_migrations = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.013388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.013472 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] max_concurrent_snapshots = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.013563 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.013659 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] max_local_block_devices = 3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.013757 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.013854 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_count = 30 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.013951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.014036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] max_logfile_size_mb = 200 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.014242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.014335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] maximum_instance_delete_attempts = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.014549 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.014648 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen = 0.0.0.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.014902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.015001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] metadata_listen_port = 8775 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.015215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.015315 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] metadata_workers = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.015536 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] migrate_max_retries = -1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.015626 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.015856 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] mkisofs_cmd = genisoimage {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.015959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.016210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.016377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] my_block_storage_ip = 10.222.0.52 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.016526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] my_ip = 10.222.0.52 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.016669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.016828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] network_allocate_retries = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.016980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.017137 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] non_inheritable_image_properties = ['cache_in_nova', 'bittorrent'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.017308 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.017474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.017625 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen = 0.0.0.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.017766 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_listen_port = 8774 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.017918 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.018084 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_unique_server_name_scope = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.018243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.018386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.018537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] osapi_compute_workers = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.018688 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.018829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] password_length = 12 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.019001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] periodic_enable = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.019144 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.019308 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] periodic_fuzzy_delay = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.019468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.019617 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.019711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] pointer_model = usbtablet {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.019939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.020037 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] preallocate_images = none {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.020246 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] publish_errors = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.020428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.020518 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] pybasedir = /opt/stack/nova {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.020603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.020795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ram_allocation_ratio = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.020982 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.021092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_burst = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.021180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.021428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_except_level = CRITICAL {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.021525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.021736 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] rate_limit_interval = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.021858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.022042 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] reboot_timeout = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.022181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.022348 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] reclaim_instance_interval = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.022509 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.022658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] record = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.022825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.022931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] reimage_timeout_per_gb = 20 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.023145 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.023249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] report_interval = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.023459 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.023559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] rescue_timeout = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.023775 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.023880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_cpus = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.024090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.024194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_disk_mb = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.024393 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] reserved_host_memory_mb = 512 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.024492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.024735 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] reserved_huge_pages = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.024848 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.025012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] resize_confirm_window = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.025107 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.025342 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] resize_fs_using_block_device = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.025450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.025655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.025747 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] resume_guests_state_on_host_boot = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.026779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.026947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] rootwrap_config = /etc/nova/rootwrap.conf {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.027145 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.027303 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_action = reap {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.027674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.027674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.027861 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_poll_interval = 1800 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.027861 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] running_deleted_instance_timeout = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.028025 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.028124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler_instance_sync_interval = 120 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.028209 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.028324 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_down_time = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.028477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.028649 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] servicegroup_driver = db {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.028790 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.028958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] shelved_offload_time = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.029105 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.029309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] shelved_poll_interval = 3600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.029457 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.029617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] shutdown_timeout = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.029773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.029929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] source_is_ipv6 = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.030091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.030236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ssl_only = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.030407 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.030556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] state_path = /opt/stack/data/nova {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.030719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.030871 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_interval = 600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.031171 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.031336 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] sync_power_state_pool_size = 1000 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.031497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.031651 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] syslog_log_facility = LOG_USER {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.031802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.031953 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] tempdir = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.032095 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.032243 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] timeout_nbd = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.032415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.032636 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] transport_url = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.032801 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.032939 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] update_resources_interval = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.033083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.033234 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] use_cow_images = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.033382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.033473 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] use_eventlog = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.033558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.033642 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] use_journal = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.033726 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.033844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] use_json = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.034030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.034140 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] use_rootwrap_daemon = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.034358 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.034452 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] use_stderr = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.034672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.034780 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] use_syslog = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.035000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.035094 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vcpu_pin_set = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.035318 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_is_fatal = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.035417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.035623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.035731 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vif_plugging_timeout = 300 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.035932 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.036034 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] virt_mkfs = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.036237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] volume_usage_poll_interval = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.036330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.036541 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] watch_log_file = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.036648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.036857 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.036963 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] web = /usr/share/spice-html5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 25 00:55:02.037163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.037306 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.disable_process_locking = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.037505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.037621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_concurrency.lock_path = /opt/stack/data/nova {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.037852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.037959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.auth_strategy = keystone {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.038168 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.038275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.compute_link_prefix = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.038478 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.038574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.config_drive_skip_versions = 1.0 2007-01-19 2007-03-01 2007-08-29 2007-10-10 2007-12-15 2008-02-01 2008-09-01 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.038798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.038891 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.dhcp_domain = novalocal {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.039120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.enable_instance_password = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.039213 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.039431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.039526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.glance_link_prefix = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.039742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_fixed_size = 100 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.039843 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.040052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.040143 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_cells_batch_strategy = distributed {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.040364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.instance_list_per_project_cells = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.040457 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.040677 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.list_records_by_skipping_down_cells = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.040770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.040989 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.041089 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.local_metadata_per_cell = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.041327 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.041435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.max_limit = 1000 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.041718 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.041899 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.metadata_cache_expiration = 15 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.042031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.042259 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.neutron_default_tenant_id = default {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.042354 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.042640 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.042804 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.use_forwarded_for = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.042950 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.043135 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.use_neutron_default_nets = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.043309 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.043456 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_connect_timeout = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.043601 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.043749 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_failure_fatal = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.043892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.044068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_read_timeout = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.044179 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.044401 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_ssl_certfile = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.044548 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.044728 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_dynamic_targets = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.044879 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.045044 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_jsonfile_path = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.045354 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.045500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api.vendordata_providers = ['StaticJSON'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.045694 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.045847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend = dogpile.cache.memcached {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.045996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.046150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.backend_argument = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.046319 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.046458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.config_prefix = cache.oslo {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.046673 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.046816 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.dead_timeout = 60.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.047019 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.debug_cache_backend = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.047179 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.047294 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_retry_client = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.047497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.047650 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.enable_socket_keepalive = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.047802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.047942 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.048090 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.enabled = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.048238 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.048379 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.expiration_time = 600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.048617 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.048761 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_attempts = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.048903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.049055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.hashclient_retry_delay = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.049205 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.049361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_dead_retry = 300 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.049520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.049671 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_password = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.049818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.049959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.050109 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.050260 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.050414 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.050554 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_maxsize = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.050655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.050743 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_pool_unused_timeout = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.050899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.050994 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_sasl_enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.051206 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.051301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_servers = ['localhost:11211'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.051513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.051636 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_socket_timeout = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.051822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.051938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.memcache_username = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.052137 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.052234 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.proxies = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.052446 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.052547 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_attempts = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.052763 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.052870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.retry_delay = 0.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.053070 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.053162 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_count = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.053422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.053561 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_idle = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.053729 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.socket_keepalive_interval = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.053831 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.054032 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.054148 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_allowed_ciphers = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.054334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.054432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.054638 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.054738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.054945 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.055036 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.055251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cache.tls_keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.055343 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.055573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_section = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.055674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.055907 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.auth_type = password {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.056004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.056220 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.056319 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.056552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.056687 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.catalog_info = volumev3::publicURL {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.056858 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.056983 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.057176 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.058163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.058163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.058492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.cross_az_attach = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.058650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.058802 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.debug = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.058899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.058899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.059080 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.endpoint_template = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.059210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.059369 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.http_retries = 3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.059543 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.059687 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.059814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.060000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.060127 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.060330 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.os_region_name = RegionOne {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.060423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.060644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.060750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.060954 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cinder.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.061054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.061301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.consecutive_build_service_disable_threshold = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.061418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.061613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_dedicated_set = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.061766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.061922 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.cpu_shared_set = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.062014 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.062240 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.image_type_exclude_list = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.062343 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.062542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.062645 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.live_migration_wait_for_vif_plug = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.062849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.062956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_concurrent_disk_ops = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.063157 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.max_disk_devices_to_attach = -1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.063247 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.063468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.063628 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.packing_host_numa_cells_allocation_strategy = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.063774 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.063881 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.provider_config_location = /etc/nova/provider_config/ {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.064105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.resource_provider_association_refresh = 300 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.064201 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.064376 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.064472 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] compute.shutdown_retry_interval = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.064695 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.064790 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] conductor.workers = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.065003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.065098 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] console.allowed_origins = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.065331 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.065441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_ciphers = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.065639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.065743 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] console.ssl_minimum_version = default {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.065951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.066046 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] consoleauth.token_ttl = 600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.066263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.066359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.066567 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.066660 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.066878 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.066985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.067184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.067333 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.067489 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.067582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.connect_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.067791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.endpoint_override = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.067913 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.068157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.068298 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.068399 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.068487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.068679 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.068784 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.max_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.068980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.min_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.069079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.069481 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.region_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.069667 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.069667 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.070450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.070603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.070746 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.service_type = accelerator {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.070890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071041 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071489 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071594 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.status_code_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.071990 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.valid_interfaces = ['internal', 'public'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] cyborg.version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072267 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072349 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.backend = sqlalchemy {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072446 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072538 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.connection = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072622 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_debug = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.072864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.073232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_parameters = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.073476 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.073659 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.073775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_recycle_time = 3600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.073935 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.connection_trace = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.074099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.074249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.db_inc_retry_interval = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.074336 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.074596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retries = 20 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.074739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.074886 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.db_max_retry_interval = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.075029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.075179 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.db_retry_interval = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.075316 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.075478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.max_overflow = 50 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.075629 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.075777 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.max_pool_size = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.075925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.076076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.max_retries = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.076225 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.076366 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_enable_ndb = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.076506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.076692 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.mysql_sql_mode = TRADITIONAL {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.076837 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.076991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.pool_timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.077132 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.077336 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.retry_interval = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.077506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.077646 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.slave_connection = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.077752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.077964 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] database.sqlite_synchronous = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.078081 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.078328 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.backend = sqlalchemy {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.078424 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.078618 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.078722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.078928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.079023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_debug = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.079237 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.079331 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_parameters = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.079545 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_recycle_time = 3600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.079651 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.079861 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.079955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.connection_trace = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.080166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_inc_retry_interval = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.080265 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.080478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retries = 20 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.080657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.080788 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_max_retry_interval = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.080883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.081096 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.db_retry_interval = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.081199 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.081444 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.081612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_overflow = 50 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.081757 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_pool_size = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.081849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.082067 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.082161 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.max_retries = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.082373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_enable_ndb = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.082474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.082685 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.082783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.mysql_sql_mode = TRADITIONAL {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.082993 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.pool_timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.083084 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.083307 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.083405 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.retry_interval = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.083609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.slave_connection = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.083707 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.083925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.084040 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] api_database.sqlite_synchronous = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.084241 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.084344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] devices.enabled_mdev_types = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.084558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.084650 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.cipher = aes-xts-plain64 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.084865 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.084970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.085176 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.085287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ephemeral_storage_encryption.key_size = 512 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.085517 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.085812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.api_servers = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.085812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.085993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.086193 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.086331 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.086498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.086656 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.086776 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.086944 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.087116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.087236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.connect_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.087430 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.087602 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.debug = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.087782 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.087991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.default_trusted_certificate_ids = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.088106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.088245 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_certificate_validation = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.089852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.enable_rbd_download = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.089981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090081 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.endpoint_override = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090457 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090533 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.max_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090809 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.min_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.090946 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.091099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.num_retries = 3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.091190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.091404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_ceph_conf = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.091553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.091715 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_connect_timeout = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.091813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.092111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_pool = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.092306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.092463 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.rbd_user = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.092582 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.092782 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.region_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.092956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.093109 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.093250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.service_type = image {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.093429 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.093586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.093737 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.093879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.094023 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.094142 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.status_code_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.094307 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.094450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.094554 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.094797 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.valid_interfaces = ['internal', 'public'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.094891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.095109 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.verify_glance_signatures = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.095219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.095418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.095511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] glance.version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.095731 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.095839 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] guestfs.debug = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.096047 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_cdrom = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.096154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.096357 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.096451 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.config_drive_inject_password = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.096671 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.dynamic_memory_ratio = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.096764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.096984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_instance_metrics_collection = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.097088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.097314 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.enable_remotefx = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.097425 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.097641 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.instances_path_share = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.097742 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.097954 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.iscsi_initiator_list = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.098126 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.098265 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.limit_cpu_features = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.098535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.098653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_count = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.098749 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.098875 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.mounted_disk_query_retry_interval = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.099079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.099240 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_check_timeframe = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.099389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.099534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.power_state_event_polling_interval = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.099724 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.099883 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.qemu_img_cmd = qemu-img.exe {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.100039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.100189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.use_multipath_io = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.100331 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.100478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_count = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.100646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.100782 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.volume_attach_retry_interval = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.100922 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.101076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.vswitch_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.101247 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.101428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] hyperv.wait_soft_reboot_seconds = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.101582 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.101737 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] mks.enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.101902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.102264 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] mks.mksproxy_base_url = http://127.0.0.1:6090/ {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.102432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.102682 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.manager_interval = 2400 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.102833 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.102982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.precache_concurrency = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.103123 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.103264 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_base_images = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.103405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.103545 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_original_minimum_age_seconds = 86400 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.103694 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.103838 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.remove_unused_resized_minimum_age_seconds = 3600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.103986 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.104157 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] image_cache.subdirectory_name = _base {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.104372 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.104580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_max_retries = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.104728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.104883 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.api_retry_interval = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.105040 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.105190 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_section = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.105331 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.105431 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.auth_type = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.105517 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.105679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.105812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.105985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.106121 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.106297 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.106437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.106603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.106751 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.106908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.connect_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.107083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.107213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.endpoint_override = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.107396 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.107524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.107714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.107828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.108028 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.108127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.max_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.108345 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.108450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.min_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.108699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.108815 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.partition_key = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.108963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.109063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.peer_list = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.109312 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.109420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.region_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.109629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.serial_console_state_timeout = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.109722 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.109937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.110039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.110257 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.110349 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.service_type = baremetal {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.110564 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.110662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.110867 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.110959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.111171 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.status_code_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.111277 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.111476 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.111583 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.111819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.valid_interfaces = ['internal', 'public'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.111913 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.112125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.112229 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ironic.version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.112435 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.112535 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.backend = nova.keymgr.conf_key_mgr.ConfKeyManager {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.112752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.112879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] key_manager.fixed_key = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.113100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.113216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.113706 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_api_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.113809 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.114018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.114127 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.114347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_endpoint_type = public {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.114451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.114684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.barbican_region_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.114785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.114948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.115096 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.115250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.115347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.115561 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.115685 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.115866 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.115960 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.116168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.116274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.116479 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.number_of_retries = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.116574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.116791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.retry_delay = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.116885 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.117102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.send_service_user_token = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.117199 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.117437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.117534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.117741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.117846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.118044 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.118138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.118345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican.verify_ssl_path = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.118478 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.118650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.118769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_section = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.118963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.119070 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.auth_type = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.119275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.119425 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.119566 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.119670 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.120393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.120490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.120711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.120805 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.121016 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.121129 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.121354 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.121465 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.121660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] barbican_service_user.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.121762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.121974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.122087 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_role_id = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.122274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.approle_secret_id = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.122376 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.122581 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.122704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.122871 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.122974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.123170 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.123265 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.123478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.123573 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.123782 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.123878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.124105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_mountpoint = secret {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.124201 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.124414 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.kv_version = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.124517 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.124724 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.namespace = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.124902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.125061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.root_token_id = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.125201 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.125404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.125564 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.125722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.ssl_ca_crt_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.125869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.126011 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.126158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.126299 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.use_ssl = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.126448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.126596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vault.vault_url = http://127.0.0.1:8200 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.126737 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.126878 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.127021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.127168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.127317 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.127474 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.127615 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.127761 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.127981 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.connect_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.128122 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.128287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.endpoint_override = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.128422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.128656 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.128828 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.129055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.129198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.129408 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.max_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.129557 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.129649 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.min_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.129733 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.129831 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.region_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.129917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.130064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.130179 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.130388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.service_type = identity {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.130530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.130697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.130798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.131000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.131099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.131482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.status_code_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.131619 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.131741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.131859 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.131979 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.valid_interfaces = ['internal', 'public'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.132090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.132218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] keystone.version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.132373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.132559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.connection_uri = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.132737 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.132897 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_mode = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.133034 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.133166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_model_extra_flags = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.133346 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.133512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.cpu_models = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.133668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.133825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_attempts = 8 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.134032 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.134141 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.device_detach_timeout = 20 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.134352 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.134444 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_cachemodes = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.134663 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.134754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.disk_prefix = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.134981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.135091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.enabled_perf_events = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.135301 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.135486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.file_backed_memory = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.136393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.136595 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.gid_maps = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.137026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.137150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_disk_discard = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.137344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.hw_machine_type = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.137440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.137670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.137769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_ceph_conf = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.137990 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_poll_interval = 15 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.138083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.138306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.138396 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_copy_timeout = 600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.138645 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.138737 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_glance_store_name = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.138949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.139062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_rbd_pool = rbd {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.139275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.139403 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_type = default {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.139580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.images_volume_group = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.139691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.139902 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_key = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.140001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.140218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_partition = -2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.140313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.140534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.inject_password = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.140700 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.140852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.141000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iscsi_iface = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.141184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.iser_use_multipath = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.141350 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.141486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_bandwidth = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.141723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.141821 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_completion_timeout = 800 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.142045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.142157 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime = 500 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.142402 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.142498 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_delay = 75 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.142697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_downtime_steps = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.142796 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.143009 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_inbound_addr = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.143104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.143329 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_auto_converge = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.143447 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.143644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_permit_post_copy = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.143759 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.143954 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_scheme = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.144050 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.144270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.144373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_timeout_action = abort {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.144581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_tunnelled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.144676 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.144892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.144999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_uri = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.145201 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.live_migration_with_native_tls = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.145343 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.145537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.max_queues = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.145633 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.145854 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.mem_stats_period_seconds = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.145950 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.146165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.146268 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_options = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.146474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.146602 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.nfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.146796 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.146920 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_aoe_discover_tries = 3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.147110 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.147218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_iser_scan_tries = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.147529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_memory_encrypted_guests = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.147632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.147844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_nvme_discover_tries = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.147988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.148158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_pcie_ports = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.148346 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.148477 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.num_volume_scan_tries = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.148681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.148791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.pmem_namespaces = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.149033 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.149150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_client_cfg = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.149363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.149523 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.quobyte_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.149676 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.149844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_connect_timeout = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.150007 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.150158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retries = 12 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.150320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.150471 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_destroy_volume_retry_interval = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.150636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.150807 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_secret_uuid = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.150977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.151654 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rbd_user = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.151976 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.realtime_scheduler_priority = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.152213 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.152427 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.remote_filesystem_transport = ssh {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.152577 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.152725 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_image_id = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.152869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.153018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_kernel_id = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.153159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.153326 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rescue_ramdisk_id = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.153477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.153628 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rng_dev_path = /dev/urandom {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.153778 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.153878 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.rx_queue_size = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.154042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.154136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_options = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.154355 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.154530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.smbfs_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.154691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.154848 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_compression = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.155007 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.155158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshot_image_format = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.155320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.155581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.snapshots_directory = /opt/stack/data/nova/instances/snapshots {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.155683 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.155898 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sparse_logical_volumes = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.155996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.156211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.156323 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.156523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.156614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_group = tss {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.156835 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.156928 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.swtpm_user = tss {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.157143 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.157248 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.sysinfo_serial = unique {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.157478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.tx_queue_size = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.157580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.157784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.157875 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.uid_maps = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.158095 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.use_virtio_for_bridges = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.158198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.158422 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.virt_type = kvm {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.158523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.158746 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear = zero {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.158838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.159061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_clear_size = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.159181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.159382 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.volume_use_multipath = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.159490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.159695 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_cache_path = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.159818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.160020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_log_path = /var/log/vstorage/%(cluster_name)s/nova.log.gz {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.160115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.160338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.160436 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_group = qemu {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.160648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.160746 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_opts = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.160970 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_perms = 0770 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.161069 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.161298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.161415 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_point_base = /opt/stack/data/nova/mnt {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.161632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.161744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.vzstorage_mount_user = stack {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.161949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.162043 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] libvirt.wait_soft_reboot_seconds = 120 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.162263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.162375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_section = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.162614 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.162713 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.auth_type = password {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.162936 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.163029 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.163302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.163537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.163652 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.163829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.163922 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.164156 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.164252 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.connect_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.164495 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.164587 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.default_floating_pool = public {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.164809 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.endpoint_override = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.164908 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.165123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.extension_sync_interval = 600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.165215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.165496 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.165596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.http_retries = 3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.165812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.165913 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.166127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.166226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.166435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.max_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.166536 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168213 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168317 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.metadata_proxy_shared_secret = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168489 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.min_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168582 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168677 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.ovs_bridge = br-int {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.physnets = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.168947 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.169030 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.region_name = RegionOne {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.169159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.169336 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_metadata_proxy = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.169534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.169679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.169812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.169982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.service_type = network {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.170117 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.170298 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.170433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.170609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.170769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.170957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.status_code_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.171140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.171230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.171349 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.171563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.valid_interfaces = ['internal', 'public'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.171656 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.171874 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] neutron.version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.172013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.172197 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] notifications.bdms_in_notifications = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.172340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.172532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] notifications.default_level = INFO {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.172667 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.172864 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notification_format = unversioned {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.173001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.173178 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] notifications.notify_on_state_change = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.173338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.173540 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] notifications.versioned_notifications_topics = ['versioned_notifications'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.173785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.173890 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] pci.alias = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.173991 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.174181 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] pci.device_spec = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.174274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.174492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] pci.report_in_placement = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.174590 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.174814 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_section = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.174908 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.175143 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.auth_type = password {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.175236 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.175449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.175556 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.175853 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.175964 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.176059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.176203 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.176410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.176521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.176894 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.connect_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.177015 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.177111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.endpoint_override = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.177205 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.177323 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.177469 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.177598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.177798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.177898 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.max_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.178127 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.178225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.min_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.178529 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.178636 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.region_name = RegionOne {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.178767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.178993 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.179327 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.179466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.service_type = placement {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.179466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.179818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.179913 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.179998 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.180121 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.status_code_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.180221 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.180317 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.180403 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.180596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.valid_interfaces = ['internal', 'public'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.180699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.180902 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] placement.version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.181012 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.181233 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.cores = 20 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.181360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.181619 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.181724 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.count_usage_from_placement = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.181968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.driver = nova.quota.DbQuotaDriver {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.182111 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.182333 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.182447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_content_bytes = 10240 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.182599 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.182828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_file_path_length = 255 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.182991 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.183113 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.183209 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.injected_files = 5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.183424 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.183526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.instances = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.183735 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.183901 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.key_pairs = 100 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.184044 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.184138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.metadata_items = 128 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.184352 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.184456 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.ram = 51200 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.184660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.recheck_quota = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.184765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.184962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.185060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_group_members = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.185286 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.185381 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] quota.server_groups = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.185597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.185703 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] rdp.enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.185919 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.186183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] rdp.html5_proxy_base_url = http://127.0.0.1:6083/ {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.186280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.186506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.186597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.discover_hosts_in_cells_interval = -1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.186817 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.186910 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.enable_isolated_aggregate_filtering = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.187123 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.187222 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.image_metadata_prefilter = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.187432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.187541 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.limit_tenants_to_placement_aggregate = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.187739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.187830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_attempts = 3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.188045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.188144 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.max_placement_results = 1000 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.188351 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.188452 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.placement_aggregate_required_for_tenants = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.188663 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_availability_zone = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.188761 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.188977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_image_type_support = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.189068 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.189310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.query_placement_for_routed_network_aggregates = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.189467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.189652 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] scheduler.workers = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.189866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.189957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.190048 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_namespace = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.190274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.190379 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.aggregate_image_properties_isolation_separator = . {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.190586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.190678 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.available_filters = ['nova.scheduler.filters.all_filters'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.190911 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.191004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.build_failure_weight_multiplier = 1000000.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.191236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cpu_weight_multiplier = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.191338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.191554 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.191644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.cross_cell_move_weight_multiplier = 1000000.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.191868 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.disk_weight_multiplier = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.191957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.192203 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.192294 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.enabled_filters = ['AvailabilityZoneFilter', 'ComputeFilter', 'ComputeCapabilitiesFilter', 'ImagePropertiesFilter', 'ServerGroupAntiAffinityFilter', 'ServerGroupAffinityFilter', 'SameHostFilter', 'DifferentHostFilter'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.192518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.192610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.host_subset_size = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.192827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.image_properties_default_architecture = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.192928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.193139 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.193229 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.io_ops_weight_multiplier = -1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.193479 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.193574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_hosts = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.193792 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.isolated_images = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.193891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.194102 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.194197 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_instances_per_host = 50 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.194410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.max_io_ops_per_host = 8 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.194502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.194725 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.pci_weight_multiplier = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.194826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.195041 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.ram_weight_multiplier = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.195140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.195352 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.restrict_isolated_hosts_to_isolated_images = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.195445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.195663 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.shuffle_best_same_weighed_hosts = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.195764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.195978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_affinity_weight_multiplier = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.196077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.196291 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.soft_anti_affinity_weight_multiplier = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.196393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.196602 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.196695 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.track_instance_changes = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.196935 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.197032 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] filter_scheduler.weight_classes = ['nova.scheduler.weights.all_weighers'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.197249 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.197361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] metrics.required = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.197585 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.197680 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_multiplier = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.197945 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.198088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_of_unavailable = -10000.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.198258 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.198432 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] metrics.weight_setting = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.198584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.198774 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.base_url = ws://127.0.0.1:6083/ {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.198879 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.199094 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.199197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.199431 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.port_range = 10000:20000 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.199524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.199750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.199901 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.proxyclient_address = 127.0.0.1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.200097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.200196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_host = 0.0.0.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.200366 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.200462 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] serial_console.serialproxy_port = 6083 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.200684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_section = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.200784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.200993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.201084 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.auth_type = password {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.201333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.201463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.201646 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.201771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.201959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.202119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.202267 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.202436 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.202576 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.202784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.202888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.send_service_user_token = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.203118 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.203213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.203438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.203531 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] service_user.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.203753 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.203846 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] spice.agent_enabled = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.204064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.204198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] spice.enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.204311 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120450) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 25 00:55:02.204645 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_base_url = http://127.0.0.1:6082/spice_auto.html {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.204961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_host = 0.0.0.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.205302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] spice.html5proxy_port = 6082 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.205632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_listen = 127.0.0.1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.205960 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] spice.server_proxyclient_address = 127.0.0.1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.206280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.baseapi = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.206592 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.cert = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.206916 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.compute = auto {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.207227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.conductor = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.207535 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] upgrade_levels.scheduler = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.207854 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_section = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.208171 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.auth_type = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.208479 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.208786 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.209098 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.209435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.209747 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.210136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.210368 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vendordata_dynamic_auth.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.210698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.api_retry_count = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.211005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.ca_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.211313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cache_prefix = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.211621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.cluster_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.211931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.connection_pool_size = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.212235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.console_delay_seconds = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.212542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.datastore_regex = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.212846 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_ip = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.213150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_password = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.214085 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_port = 443 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.214416 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.host_username = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.214762 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.215067 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.integration_bridge = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.215381 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.maximum_objects = 100 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.215741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_default_policy = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.216052 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.216358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.pbm_wsdl_location = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.216682 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_log_dir = /opt/vmware/vspc {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.216986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_proxy_uri = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.217312 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.serial_port_service_uri = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.217635 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.task_poll_interval = 0.5 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.217946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.use_linked_clone = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.218268 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_keymap = en-us {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.218582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port = 5900 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.218892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vmware.vnc_port_total = 10000 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.219239 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.auth_schemes = ['none'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.219562 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.enabled = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.220125 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_base_url = http://127.0.0.1:6080/vnc_auto.html {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.220461 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_host = 0.0.0.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.220783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.novncproxy_port = 6080 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.221120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_listen = 127.0.0.1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.221477 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.server_proxyclient_address = 127.0.0.1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.221790 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_ca_certs = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.222099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_cert = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.222473 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] vnc.vencrypt_client_key = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.222818 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_compute_service_check_for_ffu = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.223171 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_fallback_pcpu_query = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.223490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_group_policy_check_upcall = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.223805 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_libvirt_livesnapshot = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.224192 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.disable_rootwrap = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.224522 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_numa_live_migration = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.224746 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.enable_qemu_monitor_announce_self = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.225059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.ensure_libvirt_rbd_instance_dir_cleanup = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.225396 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.handle_virt_lifecycle_events = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.225711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.libvirt_disable_apic = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.226023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.never_download_image_if_on_rbd = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.226336 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.reserve_disk_resource_for_image_cache = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.226649 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_cpu_compare_on_dest = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.226962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.skip_hypervisor_version_check_on_lm = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.227272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.unified_limits_count_pcpu_as_vcpu = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.227589 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] workarounds.wait_for_vif_plugged_event_during_hard_reboot = [] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.227935 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.api_paste_config = /etc/nova/api-paste.ini {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.228258 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.client_socket_timeout = 900 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.229147 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.default_pool_size = 1000 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.229147 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.keep_alive = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.229458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.max_header_line = 16384 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.229724 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.secure_proxy_ssl_header = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.230039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_ca_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.230421 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_cert_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.230665 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.ssl_key_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.230984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.tcp_keepidle = 600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.231319 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] wsgi.wsgi_log_format = %(client_ip)s "%(request_line)s" status: %(status_code)s len: %(body_length)s time: %(wall_seconds).7f {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.231636 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] zvm.ca_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.231946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] zvm.cloud_connector_url = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.232378 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] zvm.image_tmp_path = /opt/stack/data/nova/images {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.232719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] zvm.reachable_timeout = 300 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.233155 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.233345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.233678 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.233980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_process_name = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.234341 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.234640 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.234933 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_new_defaults = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.235245 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.enforce_scope = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.235578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_default_rule = default {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.235921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.236280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.policy_file = policy.yaml {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.236581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.236888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.237195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.237530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.237841 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.238164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_section = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.238745 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.auth_type = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.239049 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.cafile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.239358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.certfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.239665 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.collect_timing = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.239971 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.240293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.connect_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.240579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_id = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.240880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.endpoint_override = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.241187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.insecure = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.241513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.keyfile = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.241816 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.max_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.242119 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.min_version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.242421 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.region_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.242723 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_name = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.243028 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.service_type = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.243338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.split_loggers = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.243640 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retries = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.243942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.status_code_retry_delay = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.244249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.timeout = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.244553 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.valid_interfaces = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.245420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_limit.version = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.245762 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.246096 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.connection_string = messaging:// {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.246405 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.enabled = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.246728 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_doc_type = notification {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.247039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_size = 10000 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.247361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.es_scroll_time = 2m {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.247670 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.filter_error_trace = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.247991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.hmac_keys = SECRET_KEY {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.248312 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.sentinel_service_name = mymaster {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.248632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.socket_timeout = 0.1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.248941 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] profiler.trace_sqlalchemy = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.249295 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.249609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.250003 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.250231 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.250541 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.250851 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.251161 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.251490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.251815 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.heartbeat_timeout_threshold = 600 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.252121 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_compression = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.252446 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.252769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.253085 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.253423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.253736 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.254065 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.254376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.254689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.255002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.255307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_bytes = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.255605 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_quroum_max_memory_length = 0 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.255919 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.256228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.256537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.256855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.257167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.257537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_ca_file = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.257867 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_cert_file = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.258177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.258503 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_key_file = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.258829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_rabbit.ssl_version = {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.259184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.driver = ['messagingv2'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.259500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.retry = -1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.259836 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.topics = ['notifications'] {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.260171 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_messaging_notifications.transport_url = **** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.261176 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.261530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.file_event_handler_interval = 1 {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.261951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] oslo_reports.log_dir = None {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 25 00:55:02.262200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi_app [-] ******************************************************************************** {{(pid=120451) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 25 00:55:02.267365 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO dbcounter [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Registered counter for database nova_api Sep 25 00:55:02.288648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_db.sqlalchemy.engines [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120450) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:55:02.295267 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writer thread running {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:55:02.325122 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO dbcounter [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Registered counter for database nova_api Sep 25 00:55:02.346189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_db.sqlalchemy.engines [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120451) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:55:02.353445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.context [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ade94c2f-42bc-4402-b847-004397ecdfd4(cell1) {{(pid=120450) load_cells /opt/stack/nova/nova/context.py:464}} Sep 25 00:55:02.353994 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writer thread running {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:55:02.358005 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:55:02.358497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:55:02.360212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:55:02.360873 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:55:02.361347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:55:02.363685 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:55:02.365914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO dbcounter [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Registered counter for database nova_cell0 Sep 25 00:55:02.370616 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO dbcounter [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Registered counter for database nova_cell1 Sep 25 00:55:02.378203 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_db.sqlalchemy.engines [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120450) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:55:02.379579 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_db.sqlalchemy.engines [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120450) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:55:02.386456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writer thread running {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:55:02.389510 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writer thread running {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:55:02.414946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.context [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Found 2 cells: 00000000-0000-0000-0000-000000000000(cell0),ade94c2f-42bc-4402-b847-004397ecdfd4(cell1) {{(pid=120451) load_cells /opt/stack/nova/nova/context.py:464}} Sep 25 00:55:02.419715 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:55:02.420194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:55:02.422112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:55:02.422882 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:55:02.423368 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:55:02.425434 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:55:02.427628 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO dbcounter [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Registered counter for database nova_cell0 Sep 25 00:55:02.432135 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO dbcounter [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Registered counter for database nova_cell1 Sep 25 00:55:02.441806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_db.sqlalchemy.engines [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120451) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:55:02.442616 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_db.sqlalchemy.engines [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120451) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:55:02.449633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writer thread running {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:55:02.450942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writer thread running {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:55:02.542134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO dbcounter [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Registered counter for database nova_cell0 Sep 25 00:55:02.549859 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_db.sqlalchemy.engines [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120450) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:55:02.554961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writer thread running {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:55:02.605248 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO dbcounter [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Registered counter for database nova_cell0 Sep 25 00:55:02.612673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_db.sqlalchemy.engines [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=120451) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 25 00:55:02.618150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writer thread running {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 25 00:55:02.879190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: WARNING keystonemiddleware.auth_token [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 25 00:55:02.886699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: WARNING keystonemiddleware.auth_token [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 25 00:55:02.942740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: WARNING keystonemiddleware.auth_token [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 25 00:55:02.950710 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: WARNING keystonemiddleware.auth_token [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 25 00:55:03.235914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: WARNING keystonemiddleware.auth_token [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 25 00:55:03.242220 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: WARNING keystonemiddleware.auth_token [None req-6f449837-44c0-441c-a14c-f911c2a88784 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 25 00:55:03.248291 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b739c259c0 pid: 120450 (default app) Sep 25 00:55:03.302388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: WARNING keystonemiddleware.auth_token [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] AuthToken middleware is set with keystone_authtoken.service_token_roles_required set to False. This is backwards compatible but deprecated behaviour. Please set this to True. Sep 25 00:55:03.308899 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: WARNING keystonemiddleware.auth_token [None req-74d0a76f-5481-4739-8b4d-cfa99c59a836 None None] Configuring www_authenticate_uri to point to the public identity endpoint is required; clients may not be able to authenticate against an admin endpoint Sep 25 00:55:03.314755 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b739c259c0 pid: 120451 (default app) Sep 25 00:55:38.614286 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:38.615391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:38.616127 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:38.616852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:39.825798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-60176253-2bf1-4ca3-897b-79eb9b61d467 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Action: 'create', calling method: .version_select at 0x7fe22e8da430>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesSnapshotsTestJSON-volume-928075641"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:55:42.176896 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-60176253-2bf1-4ca3-897b-79eb9b61d467 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 200 len: 344 microversion: 2.1 time: 3.572483 Sep 25 00:55:42.178314 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 1/1] 10.222.0.52 () {60 vars in 1155 bytes} [Sun Sep 25 00:55:38 2022] POST /compute/v2.1/os-volumes => generated 344 bytes in 3577 msecs (HTTP/1.1 200) 10 headers in 448 bytes (1 switches on core 0) Sep 25 00:55:42.200633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:42.203593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:42.209498 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:42.211150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:55:42.389658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f2e174d9-7601-4662-baa5-f7621a5fcf47 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e7ad310>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:42.528897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f2e174d9-7601-4662-baa5-f7621a5fcf47 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/d25aa64c-9866-404f-8d41-c08add4e9232" status: 200 len: 351 microversion: 2.1 time: 0.144697 Sep 25 00:55:42.538433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 2/2] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:55:42 2022] GET /compute/v2.1/os-volumes/d25aa64c-9866-404f-8d41-c08add4e9232 => generated 351 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:42.748458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-90a68d4b-30c3-4f53-a9b4-d26d61a712ef tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e92e040>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:43.683085 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-90a68d4b-30c3-4f53-a9b4-d26d61a712ef tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830" status: 200 len: 345 microversion: 2.1 time: 1.490152 Sep 25 00:55:43.683767 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 1/3] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:55:42 2022] GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 => generated 345 bytes in 1495 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:43.699516 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-964ff7e1-28ee-4930-8a75-442d143d44dc tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Action: 'create', calling method: .version_select at 0x7fe22e7ad9d0>, body: {"snapshot": {"volume_id": "720d9174-f1c0-4ed6-83a6-769d28be9830", "display_name": "tempest-VolumesSnapshotsTestJSON-Snapshot-1707851173"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:55:44.187046 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-964ff7e1-28ee-4930-8a75-442d143d44dc tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "POST /compute/v2.1/os-snapshots" status: 200 len: 287 microversion: 2.1 time: 0.492545 Sep 25 00:55:44.187811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 3/4] 10.222.0.52 () {60 vars in 1162 bytes} [Sun Sep 25 00:55:43 2022] POST /compute/v2.1/os-snapshots => generated 287 bytes in 494 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:44.200925 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0e2b51bb-35f6-4f9e-b4bf-33938cf70c87 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e79c310>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:44.272880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0e2b51bb-35f6-4f9e-b4bf-33938cf70c87 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2" status: 200 len: 287 microversion: 2.1 time: 0.076939 Sep 25 00:55:44.273927 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 2/5] 10.222.0.52 () {58 vars in 1251 bytes} [Sun Sep 25 00:55:44 2022] GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 => generated 287 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:44.285669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-05bc421f-f622-4426-96e4-c47c14a71c84 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e744b80>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:44.385722 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-05bc421f-f622-4426-96e4-c47c14a71c84 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots" status: 200 len: 290 microversion: 2.1 time: 0.103786 Sep 25 00:55:44.397541 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 4/6] 10.222.0.52 () {58 vars in 1140 bytes} [Sun Sep 25 00:55:44 2022] GET /compute/v2.1/os-snapshots => generated 290 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:44.398428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b0613886-6563-48e4-8cc7-abb5dde5ff65 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e79ca60>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:44.460195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b0613886-6563-48e4-8cc7-abb5dde5ff65 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2" status: 200 len: 287 microversion: 2.1 time: 0.065790 Sep 25 00:55:44.460772 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 3/7] 10.222.0.52 () {58 vars in 1251 bytes} [Sun Sep 25 00:55:44 2022] GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 => generated 287 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:44.564842 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c0d25181-ce05-43fa-9935-880977d4736d tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e8dadc0>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:44.643035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c0d25181-ce05-43fa-9935-880977d4736d tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/783a2e12-c768-45c2-9fe2-f5a6a01793c0" status: 200 len: 350 microversion: 2.1 time: 0.088156 Sep 25 00:55:44.644297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 5/8] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:55:44 2022] GET /compute/v2.1/os-volumes/783a2e12-c768-45c2-9fe2-f5a6a01793c0 => generated 350 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:45.474878 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-06b1da5e-9c9d-4f3b-981c-8b2fe4f97010 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e79cc10>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:45.605657 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-06b1da5e-9c9d-4f3b-981c-8b2fe4f97010 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2" status: 200 len: 287 microversion: 2.1 time: 0.134982 Sep 25 00:55:45.606886 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 4/9] 10.222.0.52 () {58 vars in 1251 bytes} [Sun Sep 25 00:55:45 2022] GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 => generated 287 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:46.625169 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c295faae-45fa-42dc-8352-0e11a36f2583 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e7ad940>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:46.711010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c295faae-45fa-42dc-8352-0e11a36f2583 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2" status: 200 len: 287 microversion: 2.1 time: 0.095393 Sep 25 00:55:46.718385 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 6/10] 10.222.0.52 () {58 vars in 1251 bytes} [Sun Sep 25 00:55:46 2022] GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 => generated 287 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:47.726852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-69ca7f1a-e0c1-47f8-bdfb-32f2b6040634 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e79c3a0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:47.833177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-69ca7f1a-e0c1-47f8-bdfb-32f2b6040634 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2" status: 200 len: 288 microversion: 2.1 time: 0.111662 Sep 25 00:55:47.834032 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 5/11] 10.222.0.52 () {58 vars in 1251 bytes} [Sun Sep 25 00:55:47 2022] GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 => generated 288 bytes in 113 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:47.848157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e63d61b0-79cf-410b-b154-57a2302e3c83 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e7ad0d0>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:47.977176 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e63d61b0-79cf-410b-b154-57a2302e3c83 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "DELETE /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2" status: 202 len: 0 microversion: 2.1 time: 0.134288 Sep 25 00:55:47.978150 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 7/12] 10.222.0.52 () {58 vars in 1254 bytes} [Sun Sep 25 00:55:47 2022] DELETE /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 => generated 0 bytes in 136 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 00:55:47.991159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3e65bb87-9787-4a87-8f01-d9d906089e61 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e7b21f0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:48.057624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3e65bb87-9787-4a87-8f01-d9d906089e61 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2" status: 200 len: 287 microversion: 2.1 time: 0.074584 Sep 25 00:55:48.058747 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 6/13] 10.222.0.52 () {58 vars in 1251 bytes} [Sun Sep 25 00:55:47 2022] GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 => generated 287 bytes in 76 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:48.595591 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1750183d-f340-45bf-89bc-b24c53378782 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e7ad940>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:48.679933 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1750183d-f340-45bf-89bc-b24c53378782 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/9eeb2aa0-f64e-4961-8f33-316dd49344a0" status: 200 len: 350 microversion: 2.1 time: 0.089176 Sep 25 00:55:48.681079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 8/14] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:55:48 2022] GET /compute/v2.1/os-volumes/9eeb2aa0-f64e-4961-8f33-316dd49344a0 => generated 350 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:55:48.696644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c5056744-b5c6-4c57-9bb7-00c2a503d435 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e7b2820>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:48.870002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c5056744-b5c6-4c57-9bb7-00c2a503d435 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.178365 Sep 25 00:55:48.881564 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 7/15] 10.222.0.52 () {58 vars in 1134 bytes} [Sun Sep 25 00:55:48 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:55:48.887587 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6975f841-f3bc-46ab-ae67-366b3f83afe6 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22f105820>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:49.043891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6975f841-f3bc-46ab-ae67-366b3f83afe6 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.162178 Sep 25 00:55:49.044462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 9/16] 10.222.0.52 () {58 vars in 1149 bytes} [Sun Sep 25 00:55:48 2022] GET /compute/v2.1/os-volumes?limit=2 => generated 693 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:49.060611 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f67212ca-dd11-47a2-a0c2-864719b9bb7a tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e79caf0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:49.072631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5b2a0844-878a-4973-a773-846dceeb3969 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e744ca0>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:49.125038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-5b2a0844-878a-4973-a773-846dceeb3969 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] HTTP exception thrown: Snapshot 4d774dea-09d5-4c3f-851e-d14e249410e2 could not be found. Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5b2a0844-878a-4973-a773-846dceeb3969 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Request method failure captured: Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 HTTP/1.1 Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cKXMdfaQuA5RO9BUmaSa_C0UKanQSApSETIedRXTWyBd198c9EMKC4OKy1qZmLtxSwXjH1-nXhgFQqM3eJc5vB_pC_Lxi7DKxRoTWdeMk1upJvW23WUlKS7QUgEL-m2IvxWE76Mo-oAGwlIu3ZD_-zQUJpFIvvG9KeArZaWZCXKRo Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 1bc1cab13abc43779d57421107671bdf Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:55:49.129251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_1bc1cab13abc43779d57421107671bdf", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/1bc1cab13abc43779d57421107671bdf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1bc1cab13abc43779d57421107671bdf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1bc1cab13abc43779d57421107671bdf"}]}] Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 1bc1cab13abc43779d57421107671bdf Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesSnapshotsTestJSON-1157253975-project Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 782f421af38b4c82a38ffc6ec1758b67 Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1157253975-project Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Snapshot 4d774dea-09d5-4c3f-851e-d14e249410e2 could not be found. Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '4d774dea-09d5-4c3f-851e-d14e249410e2'} Sep 25 00:55:49.130597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Snapshot 4d774dea-09d5-4c3f-851e-d14e249410e2 could not be found. (HTTP 404) (Request-ID: req-eb415a74-3976-44b5-b37d-254ce3fb4758) Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:49.131389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 617, in show Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get_snapshot(context, id) Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 466, in wrapper Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi _reraise(exception.SnapshotNotFound(snapshot_id=snapshot_id)) Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 464, in wrapper Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, snapshot_id, *args, **kwargs) Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 701, in get_snapshot Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi item = cinderclient(context).volume_snapshots.get(snapshot_id) Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volume_snapshots.py", line 160, in get Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._get("/snapshots/%s" % snapshot_id, "snapshot") Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 00:55:49.132413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.SnapshotNotFound: Snapshot 4d774dea-09d5-4c3f-851e-d14e249410e2 could not be found. Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 619, in show Sep 25 00:55:49.134180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Snapshot 4d774dea-09d5-4c3f-851e-d14e249410e2 could not be found. Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Snapshot 4d774dea-09d5-4c3f-851e-d14e249410e2 could not be found. Sep 25 00:55:49.136292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:49.137456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5b2a0844-878a-4973-a773-846dceeb3969 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Returning 404 to user: Snapshot 4d774dea-09d5-4c3f-851e-d14e249410e2 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:55:49.139336 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5b2a0844-878a-4973-a773-846dceeb3969 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2" status: 404 len: 111 microversion: 2.1 time: 0.072522 Sep 25 00:55:49.145227 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 10/17] 10.222.0.52 () {58 vars in 1251 bytes} [Sun Sep 25 00:55:49 2022] GET /compute/v2.1/os-snapshots/4d774dea-09d5-4c3f-851e-d14e249410e2 => generated 111 bytes in 79 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:55:49.160987 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-cc7c00d0-aebb-453a-b297-562f1deeb60e tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e7adc10>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:49.315566 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f67212ca-dd11-47a2-a0c2-864719b9bb7a tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes" status: 200 len: 1034 microversion: 2.1 time: 0.257792 Sep 25 00:55:49.318194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 8/18] 10.222.0.52 () {58 vars in 1134 bytes} [Sun Sep 25 00:55:49 2022] GET /compute/v2.1/os-volumes => generated 1034 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:55:49.328198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-cc7c00d0-aebb-453a-b297-562f1deeb60e tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830" status: 202 len: 0 microversion: 2.1 time: 0.176085 Sep 25 00:55:49.329862 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 11/19] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 00:55:49 2022] DELETE /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 => generated 0 bytes in 184 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 00:55:49.333315 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-887dcb43-3962-41ac-9f70-1842a62decc0 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e7b2670>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:49.344269 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e27c3c87-f131-4edf-bb0b-7edb696e1727 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e72b310>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:49.481715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e27c3c87-f131-4edf-bb0b-7edb696e1727 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830" status: 200 len: 344 microversion: 2.1 time: 0.146654 Sep 25 00:55:49.484619 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 12/20] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:55:49 2022] GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:55:49.611505 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-887dcb43-3962-41ac-9f70-1842a62decc0 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.282413 Sep 25 00:55:49.618584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 9/21] 10.222.0.52 () {58 vars in 1167 bytes} [Sun Sep 25 00:55:49 2022] GET /compute/v2.1/os-volumes?offset=1&limit=1 => generated 353 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:49.636032 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-88f902e3-1cf6-4e72-96d2-15cacfbe1563 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e7adb80>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:49.829267 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-88f902e3-1cf6-4e72-96d2-15cacfbe1563 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/detail?limit=2" status: 200 len: 693 microversion: 2.1 time: 0.198846 Sep 25 00:55:49.830612 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 13/22] 10.222.0.52 () {58 vars in 1170 bytes} [Sun Sep 25 00:55:49 2022] GET /compute/v2.1/os-volumes/detail?limit=2 => generated 693 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:49.852256 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9872c710-6859-4824-9085-bf66718aec3c tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e7ae310>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:50.151309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9872c710-6859-4824-9085-bf66718aec3c tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.309538 Sep 25 00:55:50.152195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 10/23] 10.222.0.52 () {58 vars in 1155 bytes} [Sun Sep 25 00:55:49 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 312 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:55:50.167294 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8203cbf0-e603-4e21-880e-ac7d1ed5eb2a tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e72bc10>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:50.379255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8203cbf0-e603-4e21-880e-ac7d1ed5eb2a tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/detail?offset=1&limit=1" status: 200 len: 353 microversion: 2.1 time: 0.217260 Sep 25 00:55:50.389561 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 14/24] 10.222.0.52 () {58 vars in 1188 bytes} [Sun Sep 25 00:55:50 2022] GET /compute/v2.1/os-volumes/detail?offset=1&limit=1 => generated 353 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:50.394742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7539daeb-89e2-4169-93fb-b3c453297de1 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] Calling method '.version_select at 0x7fe22e7ae8b0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:50.506006 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9bb6bfda-8a3b-4d43-84ce-643ef8dfcd0e tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e732040>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:50.556772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-9bb6bfda-8a3b-4d43-84ce-643ef8dfcd0e tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] HTTP exception thrown: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9bb6bfda-8a3b-4d43-84ce-643ef8dfcd0e tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Request method failure captured: Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 HTTP/1.1 Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cKXMdfaQuA5RO9BUmaSa_C0UKanQSApSETIedRXTWyBd198c9EMKC4OKy1qZmLtxSwXjH1-nXhgFQqM3eJc5vB_pC_Lxi7DKxRoTWdeMk1upJvW23WUlKS7QUgEL-m2IvxWE76Mo-oAGwlIu3ZD_-zQUJpFIvvG9KeArZaWZCXKRo Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 1bc1cab13abc43779d57421107671bdf Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:55:50.559689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_1bc1cab13abc43779d57421107671bdf", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/1bc1cab13abc43779d57421107671bdf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1bc1cab13abc43779d57421107671bdf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1bc1cab13abc43779d57421107671bdf"}]}] Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 1bc1cab13abc43779d57421107671bdf Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesSnapshotsTestJSON-1157253975-project Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 782f421af38b4c82a38ffc6ec1758b67 Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1157253975-project Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '720d9174-f1c0-4ed6-83a6-769d28be9830'} Sep 25 00:55:50.560738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. (HTTP 404) (Request-ID: req-2779f68c-4df2-4ad4-a837-6ab6aeea9dd9) Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.561382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 00:55:50.562288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 25 00:55:50.563157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.564029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.566421 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9bb6bfda-8a3b-4d43-84ce-643ef8dfcd0e tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Returning 404 to user: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:55:50.567933 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9bb6bfda-8a3b-4d43-84ce-643ef8dfcd0e tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830" status: 404 len: 109 microversion: 2.1 time: 0.074211 Sep 25 00:55:50.568990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 15/25] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:55:50 2022] GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 => generated 109 bytes in 76 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:55:50.578749 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c8585411-46d3-4910-a76e-c74920eac2b0 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e7284c0>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:50.654906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-c8585411-46d3-4910-a76e-c74920eac2b0 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] HTTP exception thrown: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.659313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7539daeb-89e2-4169-93fb-b3c453297de1 tempest-VolumesTestJSON-119929802 tempest-VolumesTestJSON-119929802-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/detail" status: 200 len: 1034 microversion: 2.1 time: 0.269545 Sep 25 00:55:50.673305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 11/26] 10.222.0.52 () {58 vars in 1155 bytes} [Sun Sep 25 00:55:50 2022] GET /compute/v2.1/os-volumes/detail => generated 1034 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c8585411-46d3-4910-a76e-c74920eac2b0 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Request method failure captured: Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 HTTP/1.1 Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cKXMdfaQuA5RO9BUmaSa_C0UKanQSApSETIedRXTWyBd198c9EMKC4OKy1qZmLtxSwXjH1-nXhgFQqM3eJc5vB_pC_Lxi7DKxRoTWdeMk1upJvW23WUlKS7QUgEL-m2IvxWE76Mo-oAGwlIu3ZD_-zQUJpFIvvG9KeArZaWZCXKRo Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 1bc1cab13abc43779d57421107671bdf Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:55:50.673493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_1bc1cab13abc43779d57421107671bdf", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/1bc1cab13abc43779d57421107671bdf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1bc1cab13abc43779d57421107671bdf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1bc1cab13abc43779d57421107671bdf"}]}] Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 1bc1cab13abc43779d57421107671bdf Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesSnapshotsTestJSON-1157253975-project Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 782f421af38b4c82a38ffc6ec1758b67 Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1157253975-project Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '720d9174-f1c0-4ed6-83a6-769d28be9830'} Sep 25 00:55:50.674739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. (HTTP 404) (Request-ID: req-55003634-1c2e-4fb4-a38f-76a47f48babe) Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.675514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 25 00:55:50.676603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 25 00:55:50.682024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c8585411-46d3-4910-a76e-c74920eac2b0 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Returning 404 to user: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c8585411-46d3-4910-a76e-c74920eac2b0 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830" status: 404 len: 109 microversion: 2.1 time: 0.088508 Sep 25 00:55:50.683185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 16/27] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 00:55:50 2022] DELETE /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 => generated 109 bytes in 97 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:55:50.684394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6c9368db-cd47-4c40-926a-12479eb01dd6 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Calling method '.version_select at 0x7fe22e7b2670>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:50.726653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-6c9368db-cd47-4c40-926a-12479eb01dd6 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] HTTP exception thrown: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6c9368db-cd47-4c40-926a-12479eb01dd6 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Request method failure captured: Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 HTTP/1.1 Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cKXMdfaQuA5RO9BUmaSa_C0UKanQSApSETIedRXTWyBd198c9EMKC4OKy1qZmLtxSwXjH1-nXhgFQqM3eJc5vB_pC_Lxi7DKxRoTWdeMk1upJvW23WUlKS7QUgEL-m2IvxWE76Mo-oAGwlIu3ZD_-zQUJpFIvvG9KeArZaWZCXKRo Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 1bc1cab13abc43779d57421107671bdf Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 00:55:50.731198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_1bc1cab13abc43779d57421107671bdf", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/1bc1cab13abc43779d57421107671bdf"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1bc1cab13abc43779d57421107671bdf"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1bc1cab13abc43779d57421107671bdf"}]}] Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 1bc1cab13abc43779d57421107671bdf Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-VolumesSnapshotsTestJSON-1157253975 Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-VolumesSnapshotsTestJSON-1157253975-project Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 782f421af38b4c82a38ffc6ec1758b67 Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-VolumesSnapshotsTestJSON-1157253975-project Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '720d9174-f1c0-4ed6-83a6-769d28be9830'} Sep 25 00:55:50.732353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. (HTTP 404) (Request-ID: req-fa3e9ca2-81f6-460e-861d-3db7e39a1327) Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.732968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 00:55:50.733862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 25 00:55:50.739988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6c9368db-cd47-4c40-926a-12479eb01dd6 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] Returning 404 to user: Volume 720d9174-f1c0-4ed6-83a6-769d28be9830 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6c9368db-cd47-4c40-926a-12479eb01dd6 tempest-VolumesSnapshotsTestJSON-1157253975 tempest-VolumesSnapshotsTestJSON-1157253975-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830" status: 404 len: 109 microversion: 2.1 time: 0.064686 Sep 25 00:55:50.742136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 12/28] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:55:50 2022] GET /compute/v2.1/os-volumes/720d9174-f1c0-4ed6-83a6-769d28be9830 => generated 109 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:55:52.397530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-493044a4-6a6f-4ab9-9d46-0c0299c94733 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: .version_select at 0x7fe22e728a60>, body: {"keypair": {"name": "tempest-keypair-1746086312", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBC5orPZTWSdKequi5KHXBCa5Tk3kOeSbH/f0XGwFCA48KCftfTzEtAWCWZ3kMZ1UZkO1ysb9aP8k8y+hsATBJ9mhcd5b25CdHIf0Xe6GAU5GCqzGH5tpf8NqzhT6LQkzaA=="}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:55:52.543827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-493044a4-6a6f-4ab9-9d46-0c0299c94733 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:55:52.572053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-493044a4-6a6f-4ab9-9d46-0c0299c94733 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for user 96386a16f0b54b7c94d32bd948caf118 and project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:55:52.669875 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-493044a4-6a6f-4ab9-9d46-0c0299c94733 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.777783 Sep 25 00:55:52.673471 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 17/29] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 00:55:51 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 782 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:55:56.591974 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-459145675", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:55:56.929152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] validate_networks() for [('e04e78f2-fc42-4b3e-9ba5-62cef58448f8', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:55:57.842183 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ad120d4d-ef8d-4fa2-bcec-64113e507364 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Action: 'create', calling method: .version_select at 0x7fe22e6845e0>, body: {"volume": {"size": 1, "display_name": "tempest-VolumesGetTestJSON-Volume-1171051222", "metadata": {"Type": "work"}}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:55:58.920060 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ad120d4d-ef8d-4fa2-bcec-64113e507364 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 200 len: 353 microversion: 2.1 time: 1.430764 Sep 25 00:55:58.921098 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 18/30] 10.222.0.52 () {60 vars in 1156 bytes} [Sun Sep 25 00:55:57 2022] POST /compute/v2.1/os-volumes => generated 353 bytes in 1433 msecs (HTTP/1.1 200) 10 headers in 448 bytes (2 switches on core 0) Sep 25 00:55:58.934698 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f5157272-2322-4ff0-9675-081fea82b8c6 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Calling method '.version_select at 0x7fe22e6baf70>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:55:59.053564 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f5157272-2322-4ff0-9675-081fea82b8c6 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f" status: 200 len: 353 microversion: 2.1 time: 0.122721 Sep 25 00:55:59.054033 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 19/31] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:55:58 2022] GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f => generated 353 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:55:59.217598 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-841749465", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5dc49b54-686c-4619-ad4c-65c72c26fd31"}], "security_groups": [{"name": "tempest-securitygroup--208384066"}], "key_name": "tempest-keypair-1746086312", "user_data": "}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:55:59.457868 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] validate_networks() for [('5dc49b54-686c-4619-ad4c-65c72c26fd31', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:56:00.890977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:56:00.892266 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:56:00.892953 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:56:00.893606 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:56:00.895572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:56:00.913313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:00.918167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:00.918696 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:00.919563 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:00.920120 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:00.920695 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:00.951753 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:56:00.956316 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for user 96386a16f0b54b7c94d32bd948caf118 and project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:56:00.975760 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:56:00.980150 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:56:01.046214 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:56:01.047224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:56:01.047860 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:56:01.048420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:56:01.048997 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:56:01.069168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:01.069692 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:01.070150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:01.070942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:01.071387 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:01.071809 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:01.109672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d6c389cf-b02d-4f6f-b38f-9c392689fe23 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.896704 Sep 25 00:56:01.114211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Getting quotas for project 28568584195d475fbc52d76185e09044. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:56:01.115514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 20/32] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:55:59 2022] POST /compute/v2.1/servers => generated 401 bytes in 1906 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 00:56:01.122794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Getting quotas for user 8eabe2a74a604c47bc2e5c88b960dcb6 and project 28568584195d475fbc52d76185e09044. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:56:01.130746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-cd2985b9-ffb7-4856-85c2-2e725a99b8ce tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Calling method '.version_select at 0x7fe22e5d1ca0>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:01.137006 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:56:01.139901 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:56:01.227790 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-cd2985b9-ffb7-4856-85c2-2e725a99b8ce tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f" status: 200 len: 354 microversion: 2.1 time: 0.100973 Sep 25 00:56:01.228643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 21/33] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:56:01 2022] GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f => generated 354 bytes in 104 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:01.277000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cd31322a-ece7-4867-8cc1-c4082899bb1b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 5.030199 Sep 25 00:56:01.281915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 13/34] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:55:56 2022] POST /compute/v2.1/servers => generated 383 bytes in 5036 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 00:56:01.303111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f5c978c8-a98a-46a2-97d5-d06bdbe781d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:01.303926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f5c978c8-a98a-46a2-97d5-d06bdbe781d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:01.440534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f5c978c8-a98a-46a2-97d5-d06bdbe781d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:01.479037 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f5c978c8-a98a-46a2-97d5-d06bdbe781d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1241 microversion: 2.1 time: 0.195362 Sep 25 00:56:01.480182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 14/35] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:01 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1241 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:01.495262 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ec88aca7-55fc-4b33-90cb-97ee2af01c47 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Calling method '.version_select at 0x7fe22e7b2280>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:01.578738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ec88aca7-55fc-4b33-90cb-97ee2af01c47 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f" status: 200 len: 354 microversion: 2.1 time: 0.091201 Sep 25 00:56:01.586182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 15/36] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:56:01 2022] GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f => generated 354 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:01.597883 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-62baa1a0-6f9a-4969-9752-fb418b83c095 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:01.598526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-62baa1a0-6f9a-4969-9752-fb418b83c095 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:01.692046 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-62baa1a0-6f9a-4969-9752-fb418b83c095 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:01.701310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-62baa1a0-6f9a-4969-9752-fb418b83c095 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1221 microversion: 2.1 time: 0.113370 Sep 25 00:56:01.701907 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 16/37] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:01 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1221 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:01.715562 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-351d262f-9b92-4977-b20f-02fd4cbc6997 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Calling method '.version_select at 0x7fe22ddea280>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:01.813920 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3da3aa3b-1d17-4d39-a8ae-2fbdbc410469 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'create', calling method: .version_select at 0x7fe22e5ed310>, body: {"keypair": {"name": "tempest-keypair-1654387516", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBDkP0K6wTm/iK0N9Zl7IxGGwoOY0zlQlEQgDXMIx4dPSXLwndS8JPQK/zZ1M0x9PuzG9xrKeojHsVh7FSfr4AOLWyFquQpCe4RSLRN0WL1Jb3RjubZzBmoof3YW2n8yeaQ=="}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:01.846798 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-351d262f-9b92-4977-b20f-02fd4cbc6997 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f" status: 202 len: 0 microversion: 2.1 time: 0.143450 Sep 25 00:56:01.853238 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 17/38] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 00:56:01 2022] DELETE /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f => generated 0 bytes in 151 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 00:56:01.860389 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fae0ba6e-d8b9-4a6b-ba70-86209fb153c7 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Calling method '.version_select at 0x7fe22ddea790>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:01.866119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-3da3aa3b-1d17-4d39-a8ae-2fbdbc410469 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Getting quotas for project f09655506e354c9d991f4df9a92f4086. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:56:01.873579 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-3da3aa3b-1d17-4d39-a8ae-2fbdbc410469 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Getting quotas for user dbe551f8c10c4111b4ad7380a6482564 and project f09655506e354c9d991f4df9a92f4086. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:56:01.930938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3da3aa3b-1d17-4d39-a8ae-2fbdbc410469 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.698420 Sep 25 00:56:01.933490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 22/39] 10.222.0.52 () {62 vars in 1200 bytes} [Sun Sep 25 00:56:01 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 704 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 25 00:56:01.967935 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fae0ba6e-d8b9-4a6b-ba70-86209fb153c7 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f" status: 200 len: 353 microversion: 2.1 time: 0.113230 Sep 25 00:56:01.968684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 18/40] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:56:01 2022] GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f => generated 353 bytes in 115 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:02.210304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-767775646", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "}]}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:02.279754 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-DeleteServersTestJSON-server-1694832930", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "networks": [{"uuid": "c3254d23-d48a-485d-a0e8-5f41e3baa079"}]}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:02.349194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] validate_networks() for [('e04e78f2-fc42-4b3e-9ba5-62cef58448f8', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:56:02.437157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] validate_networks() for [('c3254d23-d48a-485d-a0e8-5f41e3baa079', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:56:03.347093 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:56:03.348454 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:56:03.349399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:56:03.349835 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:56:03.351369 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:56:03.380037 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:03.383693 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:03.385147 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:03.386562 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:03.387596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:03.388946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:03.455038 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Getting quotas for project 28568584195d475fbc52d76185e09044. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:56:03.476074 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Getting quotas for user 8eabe2a74a604c47bc2e5c88b960dcb6 and project 28568584195d475fbc52d76185e09044. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:56:03.520380 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:56:03.524141 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:56:03.654843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0a133b34-150d-46e4-889d-73ffcbe06b0d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 383 microversion: 2.1 time: 1.450392 Sep 25 00:56:03.658087 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 19/41] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:56:02 2022] POST /compute/v2.1/servers => generated 383 bytes in 1454 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 00:56:03.670285 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:03.671540 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:03.692363 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:03.695458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:03.699501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:03.701637 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:03.703087 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:03.704845 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:03.750019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:56:03.750664 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:56:03.750966 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:56:03.751433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:56:03.751851 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:56:03.768223 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:03.770729 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:03.771393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:03.772382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:03.793950 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.019s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:03.794670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:03.934522 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Getting quotas for project 5ca92ea462d24d2d8632e5c52a3dfe63. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:56:03.951025 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Getting quotas for user f6e6706abaea4d0797af19fdb0331ead and project 5ca92ea462d24d2d8632e5c52a3dfe63. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:56:04.016950 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:56:04.025061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:56:04.233929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b63ae904-770c-4864-8fc9-f06c8d990693 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 2.268445 Sep 25 00:56:04.238495 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 23/42] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:56:01 2022] POST /compute/v2.1/servers => generated 380 bytes in 2271 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 00:56:04.262964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dfc3176e-9077-47f6-8552-c0b8c6098db0 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Calling method '.version_select at 0x7fe22e5d1700>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:04.506885 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:04.551611 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c1b4ae64-f411-4c18-97a7-0a2ac8ba5d01 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1245 microversion: 2.1 time: 0.892364 Sep 25 00:56:04.556413 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 20/43] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:03 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1245 bytes in 898 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:04.587358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f1843b01-229e-485c-8f4c-54e4c4e06411 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:04.591649 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f1843b01-229e-485c-8f4c-54e4c4e06411 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:04.627152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-dfc3176e-9077-47f6-8552-c0b8c6098db0 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] HTTP exception thrown: Volume 03cac9dc-aceb-416f-aff7-553fc2d1df0f could not be found. Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dfc3176e-9077-47f6-8552-c0b8c6098db0 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Request method failure captured: Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f HTTP/1.1 Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cd5Klm_48MYfvlqO1idE79LKf5E8qcADF7UW1unyP9YS_3kIELa0ZbNSFqrqDGFjCApRAJsbLkR7SzRnJcLuq8UKdZCe01jSRhQ7YuBfAAO656MP5mT5PBVqVjYYIlDMwjAIo3aUMvCTPhyR-1D3uqVroXJqUksMp1LaCnT7H8C_4 Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 03e39e5230b1454197fb977b74b18596 Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesGetTestJSON-447787461 Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:56:04.632035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_03e39e5230b1454197fb977b74b18596", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/03e39e5230b1454197fb977b74b18596"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/03e39e5230b1454197fb977b74b18596"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/03e39e5230b1454197fb977b74b18596"}]}] Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesGetTestJSON-447787461 Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 03e39e5230b1454197fb977b74b18596 Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesGetTestJSON-447787461 Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesGetTestJSON-447787461-project Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: a48e90a2614d4e098260868a34bb2af2 Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesGetTestJSON-447787461-project Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Volume 03cac9dc-aceb-416f-aff7-553fc2d1df0f could not be found. Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '03cac9dc-aceb-416f-aff7-553fc2d1df0f'} Sep 25 00:56:04.634100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 03cac9dc-aceb-416f-aff7-553fc2d1df0f could not be found. (HTTP 404) (Request-ID: req-b47d242a-9da3-48ac-bac9-6403d427b59a) Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:56:04.634903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 00:56:04.636052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 03cac9dc-aceb-416f-aff7-553fc2d1df0f could not be found. Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 25 00:56:04.637228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 03cac9dc-aceb-416f-aff7-553fc2d1df0f could not be found. Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 03cac9dc-aceb-416f-aff7-553fc2d1df0f could not be found. Sep 25 00:56:04.638415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:56:04.639474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dfc3176e-9077-47f6-8552-c0b8c6098db0 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] Returning 404 to user: Volume 03cac9dc-aceb-416f-aff7-553fc2d1df0f could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:56:04.640604 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-dfc3176e-9077-47f6-8552-c0b8c6098db0 tempest-VolumesGetTestJSON-447787461 tempest-VolumesGetTestJSON-447787461-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f" status: 404 len: 109 microversion: 2.1 time: 0.400913 Sep 25 00:56:04.644442 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 24/44] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 00:56:04 2022] GET /compute/v2.1/os-volumes/03cac9dc-aceb-416f-aff7-553fc2d1df0f => generated 109 bytes in 402 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:56:04.663382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c7cc8fa3-4797-4005-807d-7e28ef2eba5b tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:04.676554 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c7cc8fa3-4797-4005-807d-7e28ef2eba5b tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:05.142030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c7cc8fa3-4797-4005-807d-7e28ef2eba5b tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:05.199733 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c7cc8fa3-4797-4005-807d-7e28ef2eba5b tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1215 microversion: 2.1 time: 0.556968 Sep 25 00:56:05.205648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 25/45] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:04 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1215 bytes in 563 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:05.342050 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f1843b01-229e-485c-8f4c-54e4c4e06411 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:05.351297 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f1843b01-229e-485c-8f4c-54e4c4e06411 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1221 microversion: 2.1 time: 0.793791 Sep 25 00:56:05.370236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 21/46] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:04 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1221 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:05.586596 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:05.587955 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:05.606521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:05.613930 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:05.614989 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:05.616066 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:05.620464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:05.621187 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:06.109112 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:06.135355 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3bf7b87a-6af1-49e7-9168-9d3c9c699543 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1293 microversion: 2.1 time: 0.554923 Sep 25 00:56:06.137335 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 26/47] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:05 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1293 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:06.231439 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:06.233105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:06.243012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:06.250609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:06.251596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:06.257195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:06.258296 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:06.260391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:06.383640 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:06.384903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:06.403957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:06.404666 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:06.405359 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:06.406275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:06.407759 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:06.410742 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:06.559771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:06.568580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-cd31295b-949a-4917-a542-1eaf91d84089 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1217 microversion: 2.1 time: 0.192078 Sep 25 00:56:06.578766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 27/48] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:06 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1217 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:06.880941 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:06.904957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b692f54d-6cdc-4ab1-af22-92e1306d4753 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1211 microversion: 2.1 time: 0.676237 Sep 25 00:56:06.925206 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 22/49] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:06 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1211 bytes in 699 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:07.164733 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:07.165784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:07.176431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:07.180030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:07.180663 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:07.182728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:07.184971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:07.185695 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:07.457476 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:07.474757 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1983434793", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "334cdfa6-14bd-4f9b-b93c-dbd866cdb6bb"}], "security_groups": [{"name": "tempest-securitygroup--460209066"}], "key_name": "tempest-keypair-1654387516", "user_data": "}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:07.504718 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6fad76bd-1d49-42a7-9aa0-cf8efcd09e4b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1299 microversion: 2.1 time: 0.330441 Sep 25 00:56:07.513696 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 28/50] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:07 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1299 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:07.599919 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:07.600537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:07.612536 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:07.640229 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.028s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:07.645942 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:07.663401 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:07.664359 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:07.665163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:07.909924 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] validate_networks() for [('334cdfa6-14bd-4f9b-b93c-dbd866cdb6bb', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:56:07.968608 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:07.979869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9dc94038-9988-4e27-9fc1-7949ee784449 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1217 microversion: 2.1 time: 0.383580 Sep 25 00:56:07.991682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 29/51] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:07 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1217 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:08.012771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:08.024316 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:08.057037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:08.069382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.015s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:08.071577 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:08.072648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:08.079668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:08.084961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:08.618798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:08.628279 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-71ec9827-b3f4-4db1-9258-5b992036eff8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1211 microversion: 2.1 time: 0.633046 Sep 25 00:56:08.637579 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 30/52] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:07 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1211 bytes in 637 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:08.642866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:08.643740 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:08.665956 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:08.666634 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:08.669149 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:08.672706 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:08.673894 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:08.674801 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:09.039932 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:09.076126 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-05cc88fd-14e8-4952-8a86-cff74a6274f4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1299 microversion: 2.1 time: 0.439482 Sep 25 00:56:09.096270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 31/53] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:08 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1299 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:09.128896 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:09.131717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:09.142208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:09.142861 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:09.143612 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:09.144680 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:09.145406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:09.158480 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:09.424751 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:09.462779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-89387809-d0c0-4866-aa94-6204cbfcfe2e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1281 microversion: 2.1 time: 0.338017 Sep 25 00:56:09.466123 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 32/54] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:09 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1281 bytes in 369 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:09.654783 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:09.655462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:09.667776 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:09.670007 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:09.671304 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:09.675372 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:09.676921 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:09.677439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:09.994101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:56:09.994101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:56:09.994101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:56:09.994101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:56:09.994101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:56:10.019210 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:10.019789 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:10.033331 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:10.036935 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:10.038727 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:10.039925 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:10.041079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:10.073831 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-db9b372b-2e3f-4aa4-bda3-003395d121eb tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1267 microversion: 2.1 time: 0.423031 Sep 25 00:56:10.078109 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 33/55] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:09 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1267 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:10.117519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:10.129187 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:10.143667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Getting quotas for project f09655506e354c9d991f4df9a92f4086. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:56:10.159908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Getting quotas for user dbe551f8c10c4111b4ad7380a6482564 and project f09655506e354c9d991f4df9a92f4086. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:56:10.160892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:10.167276 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:10.173171 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:10.176567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:10.181329 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:10.189820 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:10.228729 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:56:10.240883 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:56:10.529088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-450b7c06-a252-403f-8fb7-7ac56b71692e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.20 time: 3.059043 Sep 25 00:56:10.534283 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 23/56] 10.222.0.52 () {62 vars in 1188 bytes} [Sun Sep 25 00:56:07 2022] POST /compute/v2.1/servers => generated 401 bytes in 3065 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 25 00:56:10.569150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:10.570500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:10.600847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:10.603110 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:10.604006 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:10.606500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:10.608739 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:10.615367 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:11.049084 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:11.060787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f08e35b3-1e8d-4c4b-a494-11f373907330 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1279 microversion: 2.1 time: 0.522513 Sep 25 00:56:11.080277 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 24/57] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:10 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1279 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:11.129035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-04b1db0b-8d79-4e9e-ad81-5109e06ea1ec tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:11.135249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-04b1db0b-8d79-4e9e-ad81-5109e06ea1ec tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:11.698565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-04b1db0b-8d79-4e9e-ad81-5109e06ea1ec tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:11.710065 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-04b1db0b-8d79-4e9e-ad81-5109e06ea1ec tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:11.718484 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-04b1db0b-8d79-4e9e-ad81-5109e06ea1ec tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:11.724374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-04b1db0b-8d79-4e9e-ad81-5109e06ea1ec tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1282 microversion: 2.20 time: 0.636415 Sep 25 00:56:11.743134 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 25/58] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:11 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1282 bytes in 662 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:11.771008 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:11.771697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:11.802347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:11.805573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:11.807513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:11.809246 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:11.809998 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:11.811345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:12.581212 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:12.607230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-239555d3-ffc4-4cbc-a9cf-a1847074b9af tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1273 microversion: 2.1 time: 0.862065 Sep 25 00:56:12.625187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 26/59] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:11 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1273 bytes in 872 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:12.628419 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:12.639333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:12.650983 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:12.665635 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:12.672332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:12.675491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:12.677213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:12.687224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:13.239136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:13.264501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4d85b520-2d09-496b-90c4-bc933111d9df tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1279 microversion: 2.1 time: 0.646431 Sep 25 00:56:13.284049 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 27/60] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:12 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1279 bytes in 666 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:13.304309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:13.304944 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:13.381978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:13.383232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.013s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:13.383692 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:13.385124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:13.385732 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:13.386272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:13.810407 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=4 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:14.266091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:14.301542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-081c2573-71bc-4161-ba50-94c25035264c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 4.201557 Sep 25 00:56:14.302271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 34/61] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:10 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 4204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:14.311051 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:14.330934 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:14.337855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:14.364490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-19132407-8c74-4b2c-b247-80d576de3c64 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1334 microversion: 2.20 time: 1.056060 Sep 25 00:56:14.367637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:14.368505 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 28/62] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:13 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1334 bytes in 1085 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:14.375395 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:14.393673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:14.394363 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:14.416990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:14.419194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:14.422073 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:14.424039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:14.426102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:14.430238 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:14.432464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:14.433753 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:14.434491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:14.436157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:14.440574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:14.441779 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:15.182258 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:15.191567 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d027ee02-0a25-44bc-942a-be806ff7a889 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1279 microversion: 2.1 time: 0.821190 Sep 25 00:56:15.197825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=22 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:15.220120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 29/63] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:14 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1279 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:15.328726 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:15.329671 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:15.337951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:15.341689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:15.343075 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:15.344424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:15.345135 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:15.347331 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:15.735937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:15.767826 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0187be50-a79f-4dbc-ba2e-fc56035697e5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.440612 Sep 25 00:56:15.770275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 30/64] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:15 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 447 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:15.785127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:15.786788 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:15.803334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:15.805122 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:15.807916 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:15.810741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:15.812065 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:15.813147 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:15.871214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:15.913414 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-968bfa5b-7869-42ae-8811-d458b8a0b6ca tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 1.609897 Sep 25 00:56:15.917713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 35/65] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:14 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 1615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:16.035361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:16.064374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:16.068201 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:16.070103 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2cf3eab2-dbd0-45eb-ae42-d59bde40ef53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1340 microversion: 2.20 time: 0.293651 Sep 25 00:56:16.071393 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 31/66] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:15 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1340 bytes in 300 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:16.228418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:16.229540 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:16.238762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:16.239958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:16.241573 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:16.244500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:16.249231 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:16.250422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:16.367655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:16.378130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a3f07ebe-ed65-475b-aa85-ea23b1fc186a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1279 microversion: 2.1 time: 0.154469 Sep 25 00:56:16.379050 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 36/67] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:16 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1279 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:16.792729 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:16.793684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:16.801712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:16.802500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:16.803108 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:16.810435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:16.811459 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:16.820908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:16.948901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:16.949568 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:16.959424 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:16.961066 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:16.963064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:16.964288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:16.972713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:16.973941 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:17.119346 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:17.130202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4d7d26b4-36c1-48db-9762-ec1296fee21d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.341925 Sep 25 00:56:17.131363 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 32/68] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:16 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:17.139790 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:17.140693 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:17.151769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:17.152412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:17.153130 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:17.153892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:17.154420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:17.155310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:17.303710 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:17.317619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:17.319453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:17.322088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-647fddc0-bb53-4e6f-b49f-1cd07db95d05 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1340 microversion: 2.20 time: 0.189455 Sep 25 00:56:17.325302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 33/69] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:17 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1340 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:17.527639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:17.550231 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f201a194-061b-4509-8ab6-08f417ae0884 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.615398 Sep 25 00:56:17.565345 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 37/70] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:16 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:17.578068 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:17.578693 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:17.588677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:17.589175 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:17.589657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:17.590362 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:17.596508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:17.596972 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:17.653112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-c05afc7a-ed3b-41c7-b9ee-946ae8cd13cb req-04a6e68d-bce2-48af-81fb-0a8f432816dc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "af139b53-2db6-43ea-9e69-27dfbb9304bc", "tag": "0b0017e5-0afb-4334-8450-1d4303b0d473"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:17.674345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-c05afc7a-ed3b-41c7-b9ee-946ae8cd13cb req-04a6e68d-bce2-48af-81fb-0a8f432816dc service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:17.680752 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-c05afc7a-ed3b-41c7-b9ee-946ae8cd13cb req-04a6e68d-bce2-48af-81fb-0a8f432816dc service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:17.681575 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-c05afc7a-ed3b-41c7-b9ee-946ae8cd13cb req-04a6e68d-bce2-48af-81fb-0a8f432816dc service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:17.735880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-c05afc7a-ed3b-41c7-b9ee-946ae8cd13cb req-04a6e68d-bce2-48af-81fb-0a8f432816dc service nova] Creating event network-changed:0b0017e5-0afb-4334-8450-1d4303b0d473 for instance af139b53-2db6-43ea-9e69-27dfbb9304bc on hv-cinder-maste Sep 25 00:56:17.736950 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-c05afc7a-ed3b-41c7-b9ee-946ae8cd13cb req-04a6e68d-bce2-48af-81fb-0a8f432816dc service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120451) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 25 00:56:17.737765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-c05afc7a-ed3b-41c7-b9ee-946ae8cd13cb req-04a6e68d-bce2-48af-81fb-0a8f432816dc service nova] Acquired lock "compute-rpcapi-router" {{(pid=120451) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 25 00:56:17.739836 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:17.746239 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:17.748910 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:17.753402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:17.753974 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:17.755383 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:17.771478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.compute.rpcapi [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 25 00:56:17.777411 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] Releasing lock "compute-rpcapi-router" {{(pid=120451) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 25 00:56:17.815680 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-527136ac-99d5-4345-9f21-e0e959894c22 None None] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.417823 Sep 25 00:56:17.818186 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 34/71] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:17 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:56:17.856979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:17.864888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-73f4bae9-709e-4e05-8435-86eba264675c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.296335 Sep 25 00:56:17.865487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 38/72] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:17 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:18.188149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:18.188835 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:18.195841 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:18.196319 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:18.196923 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:18.197818 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:18.198414 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:18.199055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:18.355572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:18.356454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:18.371270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:18.371777 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:18.379465 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:18.387226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:18.387824 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:18.388497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:18.608885 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:18.622935 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:18.624979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:18.626678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5bd26b22-4dff-4f7c-80d7-4d3a4bbc74fa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1340 microversion: 2.20 time: 0.276742 Sep 25 00:56:18.627628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 39/73] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:18 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1340 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:56:18.637764 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:18.655434 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fbaebb89-e2a1-4487-9092-91a9ecfb6a7c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.494403 Sep 25 00:56:18.657060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 35/74] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:18 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 500 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:18.657975 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:18.658893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:18.661336 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=83,nova_api:INSERT=8 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:18.683746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:18.685149 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:18.686026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:18.688402 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:18.693443 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:18.694204 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:18.892283 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:18.893876 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:18.908087 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:18.908597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:18.909033 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:18.909774 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:18.916542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:18.917005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:19.182303 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:19.196906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a1bee394-7625-4e41-bb7d-4c07ab0a81f8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.545471 Sep 25 00:56:19.197497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 40/75] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:18 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 559 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:19.405429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:19.443172 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3f286a2a-fa25-4adb-a837-a8bca1743dd4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.555005 Sep 25 00:56:19.447482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 36/76] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:18 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 564 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:19.665170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:19.666579 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:19.686435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:19.688822 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:19.695760 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:19.704022 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:19.705044 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:19.706036 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:19.707901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:19.708930 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:19.715267 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:19.717345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:19.728748 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:19.729557 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:19.730685 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:19.736221 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:20.004345 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:20.010033 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:20.027570 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:20.032823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:20.036643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-66457f68-28b4-4a16-96c9-ea1d09e8522d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1340 microversion: 2.20 time: 0.378372 Sep 25 00:56:20.037524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 41/77] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:19 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1340 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:20.219954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:20.220822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:20.232673 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:20.234649 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:20.243493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.009s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:20.244551 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:20.245174 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:20.245771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:20.463954 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:20.472838 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-33acbabc-6630-4818-9c75-18aa17a67bee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.796300 Sep 25 00:56:20.474558 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 37/78] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:19 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 800 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:20.491564 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:20.492896 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:20.524047 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:20.530860 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:20.532449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:20.534011 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:20.541040 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:20.543565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:21.088320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:21.103289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-33a9e731-6b13-4a83-9439-f21bf7d178b1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.884633 Sep 25 00:56:21.106619 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 42/79] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:20 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 891 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:21.121172 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:21.129718 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:21.152908 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:21.153439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:21.153878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:21.154628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:21.155083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:21.155510 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:21.181347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:21.186232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:21.203640 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-24aefb8c-0ccc-4958-aefe-6717596b076f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.726387 Sep 25 00:56:21.205702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 38/80] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:20 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:21.293093 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:21.309106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:21.311863 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:21.314255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-09790b2d-8cf8-4d1f-9a41-f000426ee1d9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1340 microversion: 2.20 time: 0.204723 Sep 25 00:56:21.330523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 43/81] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:21 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1340 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:21.391316 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-6d33f757-5789-44a0-a8e4-f8844d611107 req-092d4c15-199f-4a0f-9278-90a632d12b84 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "38e80e8e-ef14-4a77-b697-0d889f6cfa5f", "tag": "9618d105-fdf7-4a07-9a29-e94ba03b6c41"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:21.404412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=9 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:21.406751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6d33f757-5789-44a0-a8e4-f8844d611107 req-092d4c15-199f-4a0f-9278-90a632d12b84 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:21.408067 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6d33f757-5789-44a0-a8e4-f8844d611107 req-092d4c15-199f-4a0f-9278-90a632d12b84 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:21.408638 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6d33f757-5789-44a0-a8e4-f8844d611107 req-092d4c15-199f-4a0f-9278-90a632d12b84 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:21.437222 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-6d33f757-5789-44a0-a8e4-f8844d611107 req-092d4c15-199f-4a0f-9278-90a632d12b84 service nova] Creating event network-changed:9618d105-fdf7-4a07-9a29-e94ba03b6c41 for instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f on hv-cinder-maste Sep 25 00:56:21.443635 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-6d33f757-5789-44a0-a8e4-f8844d611107 req-092d4c15-199f-4a0f-9278-90a632d12b84 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062046 Sep 25 00:56:21.444459 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 39/82] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:56:21.506943 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:21.508374 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:21.519118 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:21.521021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:21.522767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:21.523760 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:21.525720 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:21.526366 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:22.131020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:22.131726 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:22.133445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:22.145437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=26 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:22.148767 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:22.150277 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:22.150776 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:22.152371 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:22.152839 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:22.153290 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:22.153765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f54bab13-d22c-4c9e-92b3-b47d724ccee6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.650850 Sep 25 00:56:22.154522 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 44/83] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:21 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:22.236511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:22.237152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:22.278414 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:22.278915 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:22.284829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:22.285925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:22.286663 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:22.287420 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:22.674972 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:22.686459 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-987f820a-8885-41e3-9214-d465f761ba17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.458842 Sep 25 00:56:22.699456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 45/84] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:22 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:22.718381 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:22.718875 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:22.749427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:22.750021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:22.753732 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:22.754605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:22.755100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:22.759366 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:22.805436 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:22.818044 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-305fd41e-bfb6-47d3-8b14-c1c7356056dd tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.693566 Sep 25 00:56:22.831446 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 40/85] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:22 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 696 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:23.112068 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:23.134552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:23.137893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:23.139830 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-22aba019-6090-4203-84c7-5eb1d17f642b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.439054 Sep 25 00:56:23.140863 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 46/86] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:22 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 441 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:23.184021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:23.185930 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:23.194406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:23.199835 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:23.200669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:23.201755 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:23.203428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:23.207621 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:23.334306 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-6d396f4e-9304-4d5c-8d4f-dfb218c8f446 req-f05493d0-5b58-4cf4-af45-e1752810e336 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "67d7a7f9-a0d9-4f96-b9b2-569049374b4f", "tag": "344eed51-ca03-4da0-b7a1-415ff8227472"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:23.369410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6d396f4e-9304-4d5c-8d4f-dfb218c8f446 req-f05493d0-5b58-4cf4-af45-e1752810e336 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:23.369822 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6d396f4e-9304-4d5c-8d4f-dfb218c8f446 req-f05493d0-5b58-4cf4-af45-e1752810e336 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:23.370395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6d396f4e-9304-4d5c-8d4f-dfb218c8f446 req-f05493d0-5b58-4cf4-af45-e1752810e336 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:23.395404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-6d396f4e-9304-4d5c-8d4f-dfb218c8f446 req-f05493d0-5b58-4cf4-af45-e1752810e336 service nova] Creating event network-changed:344eed51-ca03-4da0-b7a1-415ff8227472 for instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f on hv-cinder-maste Sep 25 00:56:23.403282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-6d396f4e-9304-4d5c-8d4f-dfb218c8f446 req-f05493d0-5b58-4cf4-af45-e1752810e336 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.427207 Sep 25 00:56:23.418343 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 41/87] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:22 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:56:23.497054 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-167c05c7-b8bb-42d0-a8f9-ab2363c23658 req-c83a89fb-6397-49d1-846e-359c5549c0cc service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "3c9017f3-8bdb-4c13-9c50-646241c8f1ba", "tag": "1f1b4ee1-4f64-49a9-bb20-210b866dfbd5"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:23.537668 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-167c05c7-b8bb-42d0-a8f9-ab2363c23658 req-c83a89fb-6397-49d1-846e-359c5549c0cc service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:23.545031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:23.556513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-167c05c7-b8bb-42d0-a8f9-ab2363c23658 req-c83a89fb-6397-49d1-846e-359c5549c0cc service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:23.557705 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-167c05c7-b8bb-42d0-a8f9-ab2363c23658 req-c83a89fb-6397-49d1-846e-359c5549c0cc service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.016s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:23.562880 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-dc2737f2-6787-4db5-b5d6-1350849be514 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.383011 Sep 25 00:56:23.582276 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 47/88] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:23 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 403 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:23.592696 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-167c05c7-b8bb-42d0-a8f9-ab2363c23658 req-c83a89fb-6397-49d1-846e-359c5549c0cc service nova] Creating event network-changed:1f1b4ee1-4f64-49a9-bb20-210b866dfbd5 for instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba on hv-cinder-maste Sep 25 00:56:23.603160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-167c05c7-b8bb-42d0-a8f9-ab2363c23658 req-c83a89fb-6397-49d1-846e-359c5549c0cc service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.112210 Sep 25 00:56:23.605830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 42/89] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:56:23.715117 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:23.715975 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:23.724330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:23.725060 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:23.725726 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:23.726586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:23.727195 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:23.728439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:23.861712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:23.862791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:23.872880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:23.874077 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:23.874723 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:23.875717 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:23.876497 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:23.877171 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:23.977399 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:23.992707 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a2b4ebf1-7538-4d32-9514-9b7ee354a3ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.279274 Sep 25 00:56:23.993517 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 48/90] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:23 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:24.059812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:24.068357 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9c01311e-80cd-4b92-86a7-8e8e8db96aa4 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.211320 Sep 25 00:56:24.070238 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 43/91] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:23 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:24.170273 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:24.170983 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:24.178806 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:24.180119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:24.180781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:24.181821 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:24.182497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:24.183095 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:24.564968 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:24.578099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:24.581518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:24.582691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4a42c3bb-0960-4575-8d47-afcf76a04ae8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.417738 Sep 25 00:56:24.583546 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 49/92] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:24 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 420 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:24.590825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:24.591447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:24.607170 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:24.607842 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:24.608358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:24.609206 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:24.611279 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:24.624756 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:24.877488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:24.888503 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-99f9fb59-7806-40fb-9261-ee03d875a16f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.302830 Sep 25 00:56:24.899492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 44/93] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:24 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:25.023630 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:25.025286 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:25.035799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:25.037077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:25.037704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:25.039724 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:25.040339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:25.040917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:25.097585 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:25.098682 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:25.108988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:25.109840 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:25.110546 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:25.111495 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:25.112100 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:25.112750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:25.255767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:25.265454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-36a1a5a9-c846-4bd0-bc06-d0f4b402111b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.249126 Sep 25 00:56:25.266165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 50/94] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:25 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:25.308597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:25.322850 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-bc6085c3-c4d6-44da-bd5e-b4e44977526a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.229705 Sep 25 00:56:25.323691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 45/95] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:25 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:25.619110 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:25.619860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:25.633820 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:25.634930 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:25.635646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:25.640946 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:25.641892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:25.642471 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:25.915400 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:25.915947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:25.930301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:25.930947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:25.931085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:25.931254 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:25.931967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:25.932425 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:25.932851 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:25.942016 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:25.943462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:25.944613 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8f9e23fa-8d26-4266-8f24-ac71fc190f22 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.329946 Sep 25 00:56:25.948636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 51/96] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:25 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:26.164612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:26.185467 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6f0fffa7-5bc9-41ee-8929-30d3928a7830 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.274674 Sep 25 00:56:26.188986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 46/97] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:25 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 279 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:26.288622 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:26.289239 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:26.295679 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:26.296295 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:26.296804 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:26.297825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:26.298345 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:26.298832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:26.346944 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:26.347991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:26.355368 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:26.356286 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:26.356883 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:26.357775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:26.358232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:26.358653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:26.450475 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:26.458242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-41882ec6-16ba-45e7-8f04-510a7f56ac9c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.174027 Sep 25 00:56:26.459108 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 52/98] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:26 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:26.506362 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:26.515868 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-dc1d0d22-7133-4174-946a-e776c02b1d34 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.173159 Sep 25 00:56:26.516571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 47/99] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:26 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:26.970243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:26.970863 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:26.977586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:26.978080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:26.978518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:26.979216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:26.979668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:26.980090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:27.123262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:27.131254 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:27.133139 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:27.134256 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9ba9b64b-797d-45e7-8214-1027cb56fb68 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.168177 Sep 25 00:56:27.134807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 53/100] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:26 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:27.211295 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:27.212091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:27.218151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:27.218931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:27.219246 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:27.220004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:27.220537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:27.221021 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:27.355847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:27.368434 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3d691328-6669-484c-a1f7-229f2baaee0d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.160974 Sep 25 00:56:27.369224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 48/101] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:27 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:27.482001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:27.482618 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:27.489478 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:27.489960 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:27.490386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:27.491684 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:27.492059 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:27.492461 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:27.540053 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:27.540806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:27.547773 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:27.548363 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:27.548875 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:27.549832 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:27.550367 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:27.550872 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:27.673544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:27.682438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ef23ef60-29e1-4467-a79f-81ea9491f5c8 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.204208 Sep 25 00:56:27.683686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 54/102] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:27 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:27.712366 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:27.720774 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-723b8307-4735-4ed1-bcc3-20dfa18d8d42 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.185243 Sep 25 00:56:27.721517 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 49/103] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:27 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:28.160910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:28.161710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:28.169060 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:28.169730 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:28.170230 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:28.171021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:28.171537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:28.172022 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:28.336675 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:28.346531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:28.348057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:28.349939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7161d0e5-2e9e-4f46-a10f-72a774ff5598 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.194162 Sep 25 00:56:28.350869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 55/104] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:28 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:28.392381 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:28.393025 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:28.395112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=8 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:28.401053 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:28.401657 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:28.402027 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:28.402733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:28.403191 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:28.403673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:28.549743 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:28.562292 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-516f434f-be1e-4f8a-b7ac-02c34848ffde tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.173931 Sep 25 00:56:28.573900 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 50/105] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:28 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:28.706954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:28.707567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:28.713711 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:28.714198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:28.714637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:28.715336 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:28.715791 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:28.716230 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:28.756371 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:28.757868 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:28.768727 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:28.770107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:28.770838 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:28.772221 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:28.773960 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:28.774816 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:28.864849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:28.875026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-688b8a2f-93e3-433e-9a37-1e64b4669b17 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.171764 Sep 25 00:56:28.876434 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 56/106] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:28 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:28.927048 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:28.935388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7f9c0d71-ff6a-49d1-b835-672593073478 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.185907 Sep 25 00:56:28.935958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 51/107] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:28 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:29.375500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:29.376439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:29.382981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:29.383551 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:29.384060 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:29.384846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:29.385399 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:29.386046 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:29.585750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:29.587114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:29.595231 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:29.598343 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:29.598343 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:29.599376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:29.602016 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:29.602571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:29.816539 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:29.828254 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-968731e6-e51a-4604-a803-a8117e60255f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.246705 Sep 25 00:56:29.829071 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 52/108] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:29 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 248 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:29.918371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:29.919918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:29.922033 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:29.931388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:29.932280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:29.933270 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:29.934462 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:29.935124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:29.935729 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:29.936520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:29.938023 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:29.939149 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3e99e5ee-0c0e-4f8a-9caf-86cf43a08337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.567598 Sep 25 00:56:29.939716 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 57/109] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:29 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 569 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:29.959661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:29.961052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:29.972954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:29.973995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:29.974733 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:29.979993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:29.980639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:29.981403 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:30.005386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:30.174299 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:30.183609 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b9743f57-ef40-4ab8-bab9-8ecc8710d6f6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.227632 Sep 25 00:56:30.191002 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 58/110] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:29 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:30.311325 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:30.321324 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-347162e3-8e02-450f-9d9d-ba3595952627 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.420190 Sep 25 00:56:30.322441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 53/111] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:29 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 428 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:30.419156 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-e6260e0a-2aed-455d-ba08-3ce073fb558c req-951de562-e70a-4850-8e63-591bb3fa77b9 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9", "tag": "96168749-3e0f-462f-9aed-adacd4fb28c0"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:30.431558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e6260e0a-2aed-455d-ba08-3ce073fb558c req-951de562-e70a-4850-8e63-591bb3fa77b9 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:30.432301 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e6260e0a-2aed-455d-ba08-3ce073fb558c req-951de562-e70a-4850-8e63-591bb3fa77b9 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:30.432735 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e6260e0a-2aed-455d-ba08-3ce073fb558c req-951de562-e70a-4850-8e63-591bb3fa77b9 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:30.452819 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-e6260e0a-2aed-455d-ba08-3ce073fb558c req-951de562-e70a-4850-8e63-591bb3fa77b9 service nova] Creating event network-changed:96168749-3e0f-462f-9aed-adacd4fb28c0 for instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 on hv-cinder-maste Sep 25 00:56:30.453633 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e6260e0a-2aed-455d-ba08-3ce073fb558c req-951de562-e70a-4850-8e63-591bb3fa77b9 service nova] Acquiring lock "compute-rpcapi-router" {{(pid=120450) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} Sep 25 00:56:30.454191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e6260e0a-2aed-455d-ba08-3ce073fb558c req-951de562-e70a-4850-8e63-591bb3fa77b9 service nova] Acquired lock "compute-rpcapi-router" {{(pid=120450) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:315}} Sep 25 00:56:30.455895 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-efe412b4-ea20-4065-adcb-106059085783 None None] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:30.456507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-efe412b4-ea20-4065-adcb-106059085783 None None] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:30.457089 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-efe412b4-ea20-4065-adcb-106059085783 None None] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:30.458032 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-efe412b4-ea20-4065-adcb-106059085783 None None] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:30.458479 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-efe412b4-ea20-4065-adcb-106059085783 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:30.459057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-efe412b4-ea20-4065-adcb-106059085783 None None] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:30.470048 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.compute.rpcapi [None req-efe412b4-ea20-4065-adcb-106059085783 None None] Automatically selected compute RPC version 6.1 from minimum service version 64 Sep 25 00:56:30.470948 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-efe412b4-ea20-4065-adcb-106059085783 None None] Releasing lock "compute-rpcapi-router" {{(pid=120450) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:333}} Sep 25 00:56:30.495198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-efe412b4-ea20-4065-adcb-106059085783 None None] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.086108 Sep 25 00:56:30.495973 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 59/112] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 88 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:30.852915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:30.853589 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:30.859556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:30.860071 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:30.860510 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:30.861204 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:30.861686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:30.862112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:30.966741 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:30.967440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:30.973875 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:30.974367 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:30.974804 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:30.975609 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:30.976087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:30.976624 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:30.993094 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:31.009289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-bc5411a5-17a8-44c9-82b2-e26d35d1f25b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.159247 Sep 25 00:56:31.009289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 54/113] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:30 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:31.167063 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:31.175742 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:31.177657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:31.179032 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f3712297-c256-4815-8634-9e312aecc32b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.216467 Sep 25 00:56:31.179620 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 60/114] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:30 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 218 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:56:31.206099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:31.206709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:31.213128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:31.213644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:31.214078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:31.214759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:31.215211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:31.215631 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:31.343643 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:31.346214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:31.346833 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:31.352415 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-bdbea9e3-3b89-4eae-86f8-730fb7bd5ee5 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.150261 Sep 25 00:56:31.353329 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 55/115] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:31 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:31.354125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:31.354641 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:31.355076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:31.355764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:31.356618 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:31.357042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:31.496967 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:31.504764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-97f2e750-dd12-40e5-a187-b33987da6ee9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.162591 Sep 25 00:56:31.505340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 61/116] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:31 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:32.032001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:32.032644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:32.038789 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:32.039272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:32.039707 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:32.040392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:32.040844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:32.041287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:32.174173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:32.182948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f45c3cd2-627a-4a2a-a9bc-43cb94dbf457 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.154808 Sep 25 00:56:32.183511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 56/117] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:32 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:32.206621 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:32.207238 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:32.213239 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:32.213847 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:32.214286 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:32.214979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:32.215432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:32.215853 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:32.376440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:32.377099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:32.383994 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:32.384486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:32.384920 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:32.385698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:32.386210 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:32.386638 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:32.420633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:32.426099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:32.443280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:32.445598 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:32.446896 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5a676377-243e-482e-a386-05f58745486a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.244204 Sep 25 00:56:32.448113 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 62/118] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:32 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:32.528482 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:32.529642 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:32.539165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:32.540281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:32.541431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:32.542606 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:32.543508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:32.544196 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:32.758681 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:32.768111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1805fa4f-a977-496e-8794-04a05367f25d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.395829 Sep 25 00:56:32.768688 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 57/119] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:32 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 397 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:33.129167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:33.139318 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-98cad094-1557-4f72-bf40-ec8d34e34013 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.614484 Sep 25 00:56:33.140461 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 63/120] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:32 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:33.207937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:33.209273 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:33.218591 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:33.219423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:33.220054 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:33.220942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:33.221624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:33.222217 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:33.488965 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:33.491624 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:33.507954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:33.513438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:33.516889 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:33.518025 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:33.518703 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:33.521760 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:33.688897 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:33.707299 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-dd28d3aa-332d-4436-911a-eab388f98ddb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.504709 Sep 25 00:56:33.708307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 58/121] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:33 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 507 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:33.747143 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:33.757314 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:33.759490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:33.761576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-37ad4a02-f62d-4275-934b-2a815531eb15 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.290915 Sep 25 00:56:33.774038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 64/122] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:33 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 304 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:33.793795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:33.794827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:33.804397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:33.804978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:33.805620 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:33.806669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:33.807122 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:33.807737 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:33.946072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:33.953870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0be86472-e9c3-41e3-ba9b-1916310bcbbc tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.165678 Sep 25 00:56:33.954432 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 59/123] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:33 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:34.171786 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:34.172713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:34.179334 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:34.179776 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:34.180207 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:34.180917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:34.181521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:34.181804 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:34.339388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:34.351867 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a09cd2db-84b3-438a-996b-57dd0985b9ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.186280 Sep 25 00:56:34.352451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 65/124] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:34 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:34.730995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:34.731596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:34.737701 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:34.738445 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:34.738925 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:34.739529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:34.739970 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:34.740386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:34.788287 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:34.796366 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:34.825172 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:34.833082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:34.835344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:34.836229 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:34.836985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:34.837939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:35.061916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:35.074501 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:35.077524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:35.083712 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0eb20446-6709-454e-abe8-ae85062aa407 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.297076 Sep 25 00:56:35.084787 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 66/125] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:34 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 301 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:56:35.091549 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:35.092587 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:35.098895 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:35.099636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:35.100377 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:35.101498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:35.102313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:35.102920 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:35.142423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:35.154412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-79ac2b5f-76e0-4fec-91ec-971be4aecab9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.427036 Sep 25 00:56:35.155398 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 60/126] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:34 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 429 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:35.338383 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:35.351431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ef76427d-dc52-4579-be16-8e3ee2d90baf tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.264162 Sep 25 00:56:35.352568 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 67/127] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:35 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 267 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:35.376089 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:35.376706 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:35.382830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:35.383323 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:35.383755 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:35.384458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:35.385954 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:35.386400 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:35.564199 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:35.580416 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-00f8a045-8799-4d1d-b22e-8c5ce3709d8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.202854 Sep 25 00:56:35.581119 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 61/128] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:35 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:36.118966 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:36.120993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:36.133280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:36.133829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:36.134276 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:36.135647 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:36.136105 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:36.136523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:36.179967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:36.180585 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:36.186994 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:36.187714 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:36.188145 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:36.189136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:36.189759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:36.190319 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:36.334252 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:36.345362 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:36.347139 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:36.348243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-cb2fc00b-f69e-4041-b043-2df5444253e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.236275 Sep 25 00:56:36.349210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 68/129] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:36 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:36.381295 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:36.383660 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:36.392216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:36.392386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:36.393033 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:36.393892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:36.395122 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:36.395780 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:36.396344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:36.402880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f2e090ed-bbfc-448c-8b2d-e72b60a52ed1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.226894 Sep 25 00:56:36.403475 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 62/130] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:36 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:36.567963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:36.577657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-aa6e0559-0594-42ef-8244-98fbebb55013 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.200827 Sep 25 00:56:36.578477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 69/131] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:36 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:36.604158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:36.604912 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:36.611537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:36.612108 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:36.612661 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:36.613711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:36.614231 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:36.614725 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:36.758881 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:36.768702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-da7ae321-291a-48e0-8a71-986470b200b0 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.168483 Sep 25 00:56:36.769523 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 63/132] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:36 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:37.377905 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:37.379299 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:37.386916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:37.387677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:37.388162 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:37.389103 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:37.389653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:37.390287 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:37.426141 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:37.427360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:37.434754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:37.435573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:37.436215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:37.453571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:37.454102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.017s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:37.454551 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:37.592041 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:37.608482 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:37.610330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:37.611962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4ca06082-e140-486f-9bb5-1ebad1e988dc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.238400 Sep 25 00:56:37.613374 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 70/133] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:37 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:37.619861 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:37.620725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:37.636939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:37.637555 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:37.641726 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:37.645297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:37.645297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:37.645297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:37.667681 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:37.680885 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5e6327d4-d56e-4267-b321-30bf6255c814 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.258379 Sep 25 00:56:37.682642 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 64/134] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:37 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:37.798377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:37.799684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:37.808067 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:37.808918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:37.809869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:37.811212 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:37.811913 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:37.812540 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:37.816698 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:37.832866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-83554b92-dcce-4364-9326-6111e1445b4d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.217164 Sep 25 00:56:37.854967 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 71/135] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:37 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:37.996013 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:38.005624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-231b72d7-8dc9-438a-8afb-4197496d0023 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.213760 Sep 25 00:56:38.007183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 65/136] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:37 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:38.638989 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:38.640466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:38.646696 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:38.647242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:38.647696 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:38.648821 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:38.649293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:38.649770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:38.705011 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:38.705666 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:38.721406 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:38.721903 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:38.722348 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:38.723056 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:38.723516 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:38.723933 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:39.105144 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:39.120390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:39.122101 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:39.123427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f62e1e05-21c8-410b-8ed5-a99446a6b8ad tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.488481 Sep 25 00:56:39.123994 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 72/137] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:38 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 490 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:39.149188 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:39.150005 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:39.161988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:39.162477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:39.162942 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:39.163679 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:39.164136 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:39.164567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:39.177767 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:39.187422 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9a3f9bdc-6ce9-4dc7-8461-2671f091b53c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.486066 Sep 25 00:56:39.188005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 66/138] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:38 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:39.202861 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:39.203642 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:39.222203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:39.222902 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:39.223500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:39.224383 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:39.225038 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:39.226153 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:39.544120 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:39.552283 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3bcf4a7e-5ed5-4e91-bfbb-f1f99c9f70e3 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.408322 Sep 25 00:56:39.570482 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 73/139] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:39 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 446 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:39.688079 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:39.705120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0e556566-3f12-43fb-8768-4d8b9373dbc6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.509874 Sep 25 00:56:39.706068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 67/140] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:39 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 512 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:40.153756 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:40.154510 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:40.160797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:40.161364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:40.161785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:40.162533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:40.163629 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:40.164325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:40.217600 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:40.220937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:40.229223 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:40.229755 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:40.230197 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:40.231079 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:40.231561 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:40.231951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:40.300066 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:40.309924 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:40.311382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:40.312587 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3537a07a-ec80-427c-b191-b94e99ad5abd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.163032 Sep 25 00:56:40.313720 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 74/141] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:40 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:40.373963 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:40.384000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e73a529d-0134-4822-808b-2d5138ccf9f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.170992 Sep 25 00:56:40.384584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 68/142] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:40 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:40.591948 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:40.593221 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:40.596544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:40.602864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:40.603354 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:40.603791 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:40.604494 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:40.604951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:40.605418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:40.626160 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:40.728958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:40.729983 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:40.736299 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:40.736788 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:40.737298 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:40.738335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:40.738849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:40.739362 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:40.792019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:40.804401 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bea25f87-9ac2-4715-90dc-d4d0736cf9e0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.216117 Sep 25 00:56:40.821880 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 75/143] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:40 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:40.892479 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:40.902853 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e1287e09-d628-4765-b725-b5f26d7f79ea tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.178359 Sep 25 00:56:40.903882 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 69/144] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:40 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:41.338198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:41.338728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:41.348410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:41.348900 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:41.349362 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:41.350934 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:41.351391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:41.351949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:41.409799 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:41.410464 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:41.418356 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:41.419237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:41.420004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:41.421350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:41.423211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:41.423942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:41.556065 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:41.567186 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:41.572654 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:41.574348 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-719b7141-1d30-4f1f-b99c-0fcccda5eb08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.240092 Sep 25 00:56:41.575618 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 76/145] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:41 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:41.614587 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:41.624584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9ddb2d97-2028-4d9a-bb75-16b4c3dd7b6d tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.221373 Sep 25 00:56:41.625156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 70/146] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:41 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:41.832283 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:41.833032 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:41.843966 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:41.844841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:41.845918 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:41.847295 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:41.847832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:41.848327 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:41.925688 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:41.926879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:41.943201 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:41.943387 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:41.944644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:41.948104 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:41.949487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:41.950467 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:42.091997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:42.104810 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d4bc8c7f-e600-4a6a-9f27-374cb4de51a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.276377 Sep 25 00:56:42.105891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 77/147] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:41 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:42.182731 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:42.188800 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-877e416d-df0b-4ea0-8376-17a63f50b2dd req-ab8673fd-313e-43c7-924a-d47257d2509a service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "af139b53-2db6-43ea-9e69-27dfbb9304bc", "name": "network-vif-plugged", "status": "completed", "tag": "0b0017e5-0afb-4334-8450-1d4303b0d473"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:42.191542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c49c5a6d-fef0-4049-b285-53a8f6ac7e99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.269936 Sep 25 00:56:42.192372 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 71/148] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:41 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:42.212010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-877e416d-df0b-4ea0-8376-17a63f50b2dd req-ab8673fd-313e-43c7-924a-d47257d2509a service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:42.212493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-877e416d-df0b-4ea0-8376-17a63f50b2dd req-ab8673fd-313e-43c7-924a-d47257d2509a service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:42.212923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-877e416d-df0b-4ea0-8376-17a63f50b2dd req-ab8673fd-313e-43c7-924a-d47257d2509a service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:42.224690 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-877e416d-df0b-4ea0-8376-17a63f50b2dd req-ab8673fd-313e-43c7-924a-d47257d2509a service nova] Creating event network-vif-plugged:0b0017e5-0afb-4334-8450-1d4303b0d473 for instance af139b53-2db6-43ea-9e69-27dfbb9304bc on hv-cinder-maste Sep 25 00:56:42.230824 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-877e416d-df0b-4ea0-8376-17a63f50b2dd req-ab8673fd-313e-43c7-924a-d47257d2509a service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.123589 Sep 25 00:56:42.231468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 78/149] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:42.607492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:42.608104 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:42.614425 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:42.614937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:42.615370 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:42.616621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:42.617036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:42.617487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:42.654597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:42.655287 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:42.661508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:42.661988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:42.662978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:42.663669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:42.664106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:42.664528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:42.747561 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:42.755908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:42.757454 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:42.758453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-60684d40-ba49-480a-a1f6-2853bd77b5d4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.154958 Sep 25 00:56:42.759168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 72/150] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:42 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:42.803668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:42.811759 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7a11fdc8-d590-470b-a9ff-36196d3c2c05 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.161178 Sep 25 00:56:42.812253 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 79/151] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:42 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:43.130365 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:43.130943 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:43.137950 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:43.138423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:43.138852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:43.139521 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:43.139966 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:43.140382 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:43.228152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:43.229163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:43.235486 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:43.235974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:43.236410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:43.237108 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:43.237593 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:43.238275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:43.278412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:43.286744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-70f2f172-71f5-4985-9465-5b99935e9736 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.160290 Sep 25 00:56:43.287405 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 73/152] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:43 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:43.378081 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:43.386361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7dab3c5d-087e-48de-880b-9c451ce83bae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.162743 Sep 25 00:56:43.386862 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 80/153] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:43 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:43.783671 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:43.784282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:43.790309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:43.790794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:43.791213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:43.791922 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:43.792373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:43.792794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:43.796704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:43.834603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:43.835363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:43.841137 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:43.841621 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:43.842053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:43.842746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:43.843198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:43.843616 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:43.937242 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:43.946026 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:43.947458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:43.948558 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0800b984-3ba4-4cd1-91a8-33cd03e1dcbe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.168773 Sep 25 00:56:43.949117 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 74/154] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:43 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:56:43.976316 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:43.985155 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1ab4bd6b-6949-4551-8e1c-8a9f6089df99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.154350 Sep 25 00:56:43.985874 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 81/155] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:43 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:44.249115 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-881054f0-acab-4b94-88bb-3d2d25cd3686 req-5d99eb9c-a54d-4147-9254-c5e8dbae64d7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "67d7a7f9-a0d9-4f96-b9b2-569049374b4f", "name": "network-vif-plugged", "status": "completed", "tag": "344eed51-ca03-4da0-b7a1-415ff8227472"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:44.258890 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-881054f0-acab-4b94-88bb-3d2d25cd3686 req-5d99eb9c-a54d-4147-9254-c5e8dbae64d7 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:44.259360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-881054f0-acab-4b94-88bb-3d2d25cd3686 req-5d99eb9c-a54d-4147-9254-c5e8dbae64d7 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:44.259787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-881054f0-acab-4b94-88bb-3d2d25cd3686 req-5d99eb9c-a54d-4147-9254-c5e8dbae64d7 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:44.270708 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-881054f0-acab-4b94-88bb-3d2d25cd3686 req-5d99eb9c-a54d-4147-9254-c5e8dbae64d7 service nova] Creating event network-vif-plugged:344eed51-ca03-4da0-b7a1-415ff8227472 for instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f on hv-cinder-maste Sep 25 00:56:44.275887 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-881054f0-acab-4b94-88bb-3d2d25cd3686 req-5d99eb9c-a54d-4147-9254-c5e8dbae64d7 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.030327 Sep 25 00:56:44.276495 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 75/156] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:44 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 32 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:44.310001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:44.310602 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:44.317543 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:44.318026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:44.318460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:44.319286 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:44.319736 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:44.320166 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:44.410566 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:44.411424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:44.419340 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:44.419822 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:44.420252 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:44.420955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:44.421534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:44.421966 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:44.458761 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:44.467249 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b28b08a1-ffd9-4b8b-a0e7-266c3b8e1e59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.160836 Sep 25 00:56:44.467846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 82/157] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:44 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:44.555765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:44.563602 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4babd516-1c55-4167-b055-024580a17f0b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.157429 Sep 25 00:56:44.564164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 76/158] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:44 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:44.976551 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:44.977188 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:44.983539 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:44.984010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:44.984439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:44.985116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:44.985588 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:44.986007 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:45.014904 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:45.015932 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:45.027827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:45.028618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:45.029334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:45.030466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:45.031205 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:45.031892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:45.152036 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:45.160290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:45.161765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:45.163165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0e4d69f6-8c2c-4bd9-9a63-d370220a1682 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.192691 Sep 25 00:56:45.163688 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 83/159] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:44 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:45.203304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:45.212052 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6d001db8-341c-467f-9d17-4875350ef5f1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.203702 Sep 25 00:56:45.212529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 77/160] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:45 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:45.497941 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:45.501672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:45.516725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:45.517399 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:45.518949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:45.521404 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:45.523669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:45.526105 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:45.587374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:45.588156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:45.604977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:45.604977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:45.604977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:45.606750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:45.608573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:45.610185 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:45.786904 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:45.797247 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0a482d78-6b63-403f-b6d2-c047bca9e54c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.213608 Sep 25 00:56:45.798110 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 78/161] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:45 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:46.128388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:46.137442 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8b77dc18-c531-47e2-87e9-94de0106b7d6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.649920 Sep 25 00:56:46.138261 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 84/162] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:45 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 652 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:46.189779 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:46.190721 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:46.200384 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:46.201612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:46.202200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:46.203072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:46.203674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:46.204262 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:46.235162 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:46.235778 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:46.244825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:46.249752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:46.250438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:46.251360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:46.251957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:46.255611 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:46.430818 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:46.440182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:46.442375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:46.444658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6909e42f-6e36-4dd0-9de3-b4da26addf5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.259409 Sep 25 00:56:46.460776 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:46.461906 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 79/163] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:46 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:46.471120 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1027715a-a1bd-4978-9501-7b3623f19af4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.240437 Sep 25 00:56:46.472514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 85/164] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:46 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:46.822547 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:46.823942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:46.833072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:46.835526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:46.837521 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:46.838227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:46.838686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:46.839113 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:47.123823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-cadf076f-1453-46a8-a26f-689c32e19927 req-becf3064-8229-49a2-9580-cebffe65850d service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c9017f3-8bdb-4c13-9c50-646241c8f1ba", "name": "network-vif-plugged", "status": "completed", "tag": "1f1b4ee1-4f64-49a9-bb20-210b866dfbd5"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:47.128764 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:47.143483 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a02b7b4d-2cd0-436b-ab34-0e715e739681 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.324971 Sep 25 00:56:47.145602 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 80/165] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:46 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:47.166439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-cadf076f-1453-46a8-a26f-689c32e19927 req-becf3064-8229-49a2-9580-cebffe65850d service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:47.167323 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-cadf076f-1453-46a8-a26f-689c32e19927 req-becf3064-8229-49a2-9580-cebffe65850d service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:47.167475 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-cadf076f-1453-46a8-a26f-689c32e19927 req-becf3064-8229-49a2-9580-cebffe65850d service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:47.173203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:47.175573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:47.201635 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:47.203030 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:47.203746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-cadf076f-1453-46a8-a26f-689c32e19927 req-becf3064-8229-49a2-9580-cebffe65850d service nova] Creating event network-vif-plugged:1f1b4ee1-4f64-49a9-bb20-210b866dfbd5 for instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba on hv-cinder-maste Sep 25 00:56:47.204441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:47.205965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:47.207062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:47.208339 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:47.211505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-cadf076f-1453-46a8-a26f-689c32e19927 req-becf3064-8229-49a2-9580-cebffe65850d service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.091571 Sep 25 00:56:47.212108 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 86/166] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:47 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:47.485725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:47.486279 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:47.507701 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:47.508203 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:47.508645 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:47.514101 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:47.514481 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:47.514919 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:47.616488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:47.627722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d05481ae-98e1-4cc2-8a3d-318d861afa82 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.468468 Sep 25 00:56:47.633323 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 81/167] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:47 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 473 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:47.644373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:47.645009 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:47.660541 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:47.660776 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:47.661199 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:47.664641 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:47.665133 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:47.665590 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:47.816930 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:47.829002 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:47.830703 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:47.832121 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a4ecc029-6257-4745-8432-7300da573923 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.357132 Sep 25 00:56:47.833378 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 87/168] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:47 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 359 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:47.912015 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:47.924287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d3d0a632-7fb8-49ad-a31e-54e4a0265cbb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1366 microversion: 2.1 time: 0.291935 Sep 25 00:56:47.925427 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 82/169] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:47 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1366 bytes in 295 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:48.169962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:48.170419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:48.189220 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:48.190004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:48.191361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:48.192109 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:48.192624 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:48.193055 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:48.432508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:48.454846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5556886d-631a-4dfc-b3ea-d068903abc22 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1321 microversion: 2.1 time: 0.289276 Sep 25 00:56:48.457464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 88/170] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:48 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1321 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:48.653315 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:48.654440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:48.661389 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:48.662819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:48.663668 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:48.664796 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:48.665662 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:48.666489 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:48.810683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:48.821240 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2682563f-8a7e-4305-9273-4b8815ee5067 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.172658 Sep 25 00:56:48.821855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 83/171] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:48 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:48.865044 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:48.867027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:48.888854 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=98 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:48.894037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:48.894665 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:48.895192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:48.897566 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:48.899111 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:48.901427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:48.951029 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:48.952203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:48.971529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:48.972283 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:48.972986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:48.974345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:48.975675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:48.976302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.185964 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:49.196327 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ce65bcfe-3134-45d7-8237-aae5787a579a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1555 microversion: 2.1 time: 0.251034 Sep 25 00:56:49.197197 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 84/172] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:48 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1555 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:49.219505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:49.220548 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:49.220947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:49.228847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:49.229074 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:49.230017 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:49.230433 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.231199 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:49.231667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:49.232102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.234874 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:49.236004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1ebbcafa-3a4e-4d8d-919a-d300cdcbab12 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.376032 Sep 25 00:56:49.239712 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 89/173] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:48 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 380 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:49.367791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8781d4ae-1d49-40b5-a836-59d8bde0b3dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.151157 Sep 25 00:56:49.373395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 85/174] 10.222.0.52 () {58 vars in 1275 bytes} [Sun Sep 25 00:56:49 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-interface => generated 284 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:56:49.495064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:49.503162 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:49.515081 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:49.515787 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:49.516455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.517460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:49.518126 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:49.518772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.707066 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:49.719422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d60960c2-cc1c-469d-b499-65d5eee3899a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1515 microversion: 2.1 time: 0.242651 Sep 25 00:56:49.721405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 90/175] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:49 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1515 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:49.744609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:49.748246 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:49.756611 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:49.757565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:49.758264 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.759274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:49.760022 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:49.760679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.799815 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lazy-loading 'flavor' on Instance uuid 67d7a7f9-a0d9-4f96-b9b2-569049374b4f {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:56:49.846917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:49.848906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:49.862458 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:49.863394 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:49.863826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.865071 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:49.866083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:49.866763 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:49.999254 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7edec1e4-dd0a-4bd4-b683-dd7f96af5f9a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f/action" status: 200 len: 32 microversion: 2.1 time: 0.258947 Sep 25 00:56:50.004271 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 86/176] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 00:56:49 2022] POST /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f/action => generated 32 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 25 00:56:50.011173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:50.012087 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:50.018892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:50.019599 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:50.020280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:50.021152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:50.021924 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:50.022697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:50.125646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:50.135079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d778ae52-1ce5-4b7a-9548-4d55a22ba274 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.294849 Sep 25 00:56:50.136107 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 91/177] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:49 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:50.267635 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:50.273721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:50.284348 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:50.289484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:50.289944 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:50.291418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:50.304380 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:50.306043 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:50.445922 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:50.460864 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-752f4001-00c8-4047-a23e-04381cb01be4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.453502 Sep 25 00:56:50.463486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 87/178] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:50 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 458 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:50.698614 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:50.710131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:50.712371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:50.716099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ab80605e-ffdf-4977-afc1-114eadd42d46 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.452856 Sep 25 00:56:50.717686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 92/179] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:50 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 456 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:56:50.903385 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-1f74f6d6-5f01-4524-95f9-cde7efd9fe55 req-9b96b09c-8878-4e01-ac91-207376dbe29f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9", "name": "network-vif-plugged", "status": "completed", "tag": "96168749-3e0f-462f-9aed-adacd4fb28c0"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:50.931159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1f74f6d6-5f01-4524-95f9-cde7efd9fe55 req-9b96b09c-8878-4e01-ac91-207376dbe29f service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:50.931542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1f74f6d6-5f01-4524-95f9-cde7efd9fe55 req-9b96b09c-8878-4e01-ac91-207376dbe29f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:50.931971 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1f74f6d6-5f01-4524-95f9-cde7efd9fe55 req-9b96b09c-8878-4e01-ac91-207376dbe29f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:50.960806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-1f74f6d6-5f01-4524-95f9-cde7efd9fe55 req-9b96b09c-8878-4e01-ac91-207376dbe29f service nova] Creating event network-vif-plugged:96168749-3e0f-462f-9aed-adacd4fb28c0 for instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 on hv-cinder-maste Sep 25 00:56:50.982501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-1f74f6d6-5f01-4524-95f9-cde7efd9fe55 req-9b96b09c-8878-4e01-ac91-207376dbe29f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.076851 Sep 25 00:56:50.988114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 88/180] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:56:51.167361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:51.169964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:51.190145 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:51.190872 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:51.192295 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:51.193960 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:51.194748 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:51.195386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:51.250826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:51.378227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-6516752e-fcdb-4b49-8cb1-0491dd65490d req-1114934c-525e-4e82-b2c3-7bd8e9a29bf8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "af139b53-2db6-43ea-9e69-27dfbb9304bc", "tag": "0b0017e5-0afb-4334-8450-1d4303b0d473"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:51.386277 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:51.404408 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6516752e-fcdb-4b49-8cb1-0491dd65490d req-1114934c-525e-4e82-b2c3-7bd8e9a29bf8 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:51.404892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6516752e-fcdb-4b49-8cb1-0491dd65490d req-1114934c-525e-4e82-b2c3-7bd8e9a29bf8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:51.405358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6516752e-fcdb-4b49-8cb1-0491dd65490d req-1114934c-525e-4e82-b2c3-7bd8e9a29bf8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:51.420934 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-6516752e-fcdb-4b49-8cb1-0491dd65490d req-1114934c-525e-4e82-b2c3-7bd8e9a29bf8 service nova] Creating event network-changed:0b0017e5-0afb-4334-8450-1d4303b0d473 for instance af139b53-2db6-43ea-9e69-27dfbb9304bc on hv-cinder-maste Sep 25 00:56:51.425950 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-6516752e-fcdb-4b49-8cb1-0491dd65490d req-1114934c-525e-4e82-b2c3-7bd8e9a29bf8 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051916 Sep 25 00:56:51.432417 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 89/181] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:51 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:56:51.500166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:51.505407 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:51.536939 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:51.540888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:51.542337 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:51.545785 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:51.548472 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:51.549630 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:52.301671 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:52.310140 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ec3d3946-6289-468e-a499-5af241c9f2f2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.825583 Sep 25 00:56:52.325764 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 90/182] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:51 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 842 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:52.337486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:52.341563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:52.352111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:52.353940 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:52.354527 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:52.355412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:52.356208 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:52.356782 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:52.384459 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:56:52.495416 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:52.512630 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-89f2711a-c160-4a95-b353-79369de07278 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 1.353461 Sep 25 00:56:52.517582 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 93/183] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:51 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 1362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:52.708719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:52.723601 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:52.725449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:52.726745 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-21429512-4ce1-4e3b-a3fd-c0b42921e9a5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.399595 Sep 25 00:56:52.734201 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 91/184] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:52 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 407 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:53.227699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-556fa81c-6a77-4a5d-8684-3dba2d01e9be req-05796a00-149b-426f-8f8d-3d2b0e641365 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38e80e8e-ef14-4a77-b697-0d889f6cfa5f", "name": "network-vif-plugged", "status": "completed", "tag": "9618d105-fdf7-4a07-9a29-e94ba03b6c41"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:53.245297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-556fa81c-6a77-4a5d-8684-3dba2d01e9be req-05796a00-149b-426f-8f8d-3d2b0e641365 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:53.247304 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-556fa81c-6a77-4a5d-8684-3dba2d01e9be req-05796a00-149b-426f-8f8d-3d2b0e641365 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:53.252339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-556fa81c-6a77-4a5d-8684-3dba2d01e9be req-05796a00-149b-426f-8f8d-3d2b0e641365 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:53.319548 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-556fa81c-6a77-4a5d-8684-3dba2d01e9be req-05796a00-149b-426f-8f8d-3d2b0e641365 service nova] Creating event network-vif-plugged:9618d105-fdf7-4a07-9a29-e94ba03b6c41 for instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f on hv-cinder-maste Sep 25 00:56:53.326599 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-556fa81c-6a77-4a5d-8684-3dba2d01e9be req-05796a00-149b-426f-8f8d-3d2b0e641365 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.102819 Sep 25 00:56:53.329236 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 94/185] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:53.340712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:53.341360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:53.358207 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:53.358892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:53.362095 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:53.363367 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:53.365523 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:53.366440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:53.554333 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:53.555997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:53.593210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:53.595390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.011s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:53.597506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:53.613392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:53.616913 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:53.618084 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:53.699612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:53.708291 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-50676398-40b8-47f4-84b2-8dbe093eee2f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.371362 Sep 25 00:56:53.710951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 92/186] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:53 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 375 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:53.761449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:53.762697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:53.776189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:53.777006 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:53.777625 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:53.778369 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:53.778847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:53.779257 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:54.056558 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:54.069596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:54.072073 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:54.073680 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fdf92cab-30de-4461-9ac4-dd765c42882e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1407 microversion: 2.20 time: 0.317043 Sep 25 00:56:54.093582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 93/187] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:53 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1407 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:54.095458 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:54.106908 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a73792c0-15c1-480e-a038-cdee94efd0e1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.568752 Sep 25 00:56:54.108143 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 95/188] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:53 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 572 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:54.738265 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:54.738889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:54.752567 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:54.753302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:54.754069 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:54.756512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:54.757149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:54.757746 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:54.986038 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:54.994821 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e794d5a4-1b7f-4b5a-8aad-11997ef4f9a9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.262452 Sep 25 00:56:55.009942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 94/189] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:54 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:55.102469 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:55.104331 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:55.113758 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:55.114427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:55.115173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:55.116372 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:55.117431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:55.118514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:55.136494 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:55.137429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:55.148672 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:55.150985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:55.152350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:55.153127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:55.155351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:55.155351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:55.510020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:55.519907 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b035606c-099a-4e80-8898-b1748dda421d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1315 microversion: 2.1 time: 0.389892 Sep 25 00:56:55.524094 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 95/190] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:55 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1315 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:55.745413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:55.760708 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:55.765451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:55.768208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c5246d68-cf7d-4971-b22f-def99dc86f16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1600 microversion: 2.20 time: 0.673176 Sep 25 00:56:55.798770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 96/191] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:56:55 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1600 bytes in 704 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:56:55.816001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:55.817218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:55.829294 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:55.830482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:55.831259 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:55.832394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:55.834791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:55.835668 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:56.009059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-59723a81-242a-4153-9a77-befd7d7bfd93 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-interface" status: 200 len: 285 microversion: 2.20 time: 0.197215 Sep 25 00:56:56.025674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 96/192] 10.222.0.52 () {60 vars in 1316 bytes} [Sun Sep 25 00:56:55 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-interface => generated 285 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:56:56.028479 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:56.029129 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:56.039984 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:56.040486 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:56.040932 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:56.043976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:56.044456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:56.044883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:56.519840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:56.545509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:56.546360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:56.547716 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d7b04c07-ce65-4096-88f8-f55075f9a5b1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.525166 Sep 25 00:56:56.549730 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 97/193] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:56 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:56.556187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:56.556681 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:56.562159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:56.562998 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:56.567414 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:56.569924 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:56.840956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:56.853488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e193e2f6-b682-4e53-9899-53f77146e3a1 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1502 microversion: 2.1 time: 0.311831 Sep 25 00:56:56.854755 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 97/194] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:56 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1502 bytes in 314 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:56:57.575579 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:57.577433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:57.588478 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:57.589328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:57.589983 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:57.590883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:57.591929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:57.593070 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:58.067757 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:58.084696 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-dd91ae13-ca8c-4129-a7a6-97420b6b3bee tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.517042 Sep 25 00:56:58.085329 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 98/195] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:57 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 518 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:58.408111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-dc29f302-4a35-4d65-9aa1-782744e3f2aa req-e14d51af-0d17-4091-aa4d-08220dd1a125 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9", "tag": "96168749-3e0f-462f-9aed-adacd4fb28c0"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:58.426721 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dc29f302-4a35-4d65-9aa1-782744e3f2aa req-e14d51af-0d17-4091-aa4d-08220dd1a125 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:58.427202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dc29f302-4a35-4d65-9aa1-782744e3f2aa req-e14d51af-0d17-4091-aa4d-08220dd1a125 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:58.428224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dc29f302-4a35-4d65-9aa1-782744e3f2aa req-e14d51af-0d17-4091-aa4d-08220dd1a125 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:58.441102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-dc29f302-4a35-4d65-9aa1-782744e3f2aa req-e14d51af-0d17-4091-aa4d-08220dd1a125 service nova] Creating event network-changed:96168749-3e0f-462f-9aed-adacd4fb28c0 for instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 on hv-cinder-maste Sep 25 00:56:58.459061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-dc29f302-4a35-4d65-9aa1-782744e3f2aa req-e14d51af-0d17-4091-aa4d-08220dd1a125 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.052234 Sep 25 00:56:58.459061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 98/196] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:56:58 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:56:58.545590 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "26138948-dd2e-489b-b1c6-31ee300ca006"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:56:58.548706 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:58.567794 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:58.570795 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:58.572547 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:58.575455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:58.576952 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:58.581019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:59.112420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:56:59.114242 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:56:59.128256 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:59.132215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:59.133035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:59.136254 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:56:59.136965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:56:59.138219 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:56:59.606939 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:56:59.620031 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e6765289-8143-427d-a518-9871ec1c7b2a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.512593 Sep 25 00:56:59.620969 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 99/197] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:56:59 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 515 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:56:59.792597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-06e24b31-2a49-4a09-89e3-a2187f9adf59 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "POST /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.252674 Sep 25 00:56:59.793544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 99/198] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 00:56:58 2022] POST /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f/os-volume_attachments => generated 194 bytes in 1255 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:57:00.648509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:00.649123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:00.655724 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:00.656421 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:00.657062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:00.658330 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:00.659075 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:00.659685 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:00.857049 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:00.890633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-25717338-5d28-495f-85cf-31095098ad7e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.246354 Sep 25 00:57:00.897738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 100/199] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:00 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:01.930306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:01.931133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:01.938714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:01.939398 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:01.939935 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:01.940721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:01.941470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:01.941898 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:02.111891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:02.121215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a50f1360-bc6c-4232-bcef-1def481ee180 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.196366 Sep 25 00:57:02.121850 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 100/200] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:01 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:57:03.145614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:03.146379 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:03.152391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:03.152871 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:03.153327 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:03.154007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:03.154452 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:03.154866 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:03.294105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:03.301827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6befcbf9-6504-4bb8-9907-82eccff3d760 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.160066 Sep 25 00:57:03.302947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 101/201] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:03 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:04.324485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:04.325170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:04.332069 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:04.336208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:04.336208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:04.336208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:04.336208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:04.337154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:04.505297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:04.513542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-645e7a4e-f664-4c39-b48b-f65b67098a89 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.192960 Sep 25 00:57:04.514110 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 101/202] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:04 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:05.536040 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:05.537311 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:05.544437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:05.544948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:05.546581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:05.547674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:05.548134 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:05.548817 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:05.696331 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:05.707392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b43bab62-c94d-4cdf-bc7f-31d4ff783e2b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.175122 Sep 25 00:57:05.708901 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 102/203] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:05 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:06.732051 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:06.732986 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:06.741300 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:06.741977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:06.742281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:06.743142 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:06.743771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:06.744198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:06.904313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:06.921723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-cc7f07bc-8eb3-46f8-8690-f68a6b11726f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.193283 Sep 25 00:57:06.936870 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 102/204] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:06 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:07.948994 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:07.950699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:07.960459 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:07.961892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:07.962572 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:07.964382 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:07.965938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:07.966975 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:08.198377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:08.211162 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5bbc31fa-86d5-4702-aeff-cdccccb9765e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.266413 Sep 25 00:57:08.212345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 103/205] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:07 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:09.235409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:09.236138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:09.243409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:09.243781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:09.244495 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:09.244926 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:09.246022 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:09.246676 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:09.475682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:09.487245 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c26679af-05e6-45dd-9313-a2a9cfc64648 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.255315 Sep 25 00:57:09.488756 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 103/206] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:09 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 258 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:09.940941 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:09.941486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:09.947775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:09.948259 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:09.949055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:09.950036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:09.950498 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:09.950929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:09.981138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 00:57:10.113623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-44da9c58-8054-4b66-85f8-78d80b0c2463 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "DELETE /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 204 len: 0 microversion: 2.1 time: 0.175865 Sep 25 00:57:10.114230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 104/207] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 00:57:09 2022] DELETE /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 00:57:10.122835 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:10.123713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:10.129647 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:10.130310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:10.131071 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:10.131802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:10.132255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:10.132669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:10.327619 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:10.338412 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5a63f2ac-72ac-4051-b06a-936fd5e59b7f tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1554 microversion: 2.1 time: 0.219656 Sep 25 00:57:10.339306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 104/208] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:10 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1554 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:10.525690 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:10.526511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:10.535318 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:10.535913 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:10.536594 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:10.537551 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:10.538045 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:10.538503 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:10.686675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:10.700293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-44dfcc62-4b45-4990-8f53-c67cdcaad27d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.183657 Sep 25 00:57:10.700878 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 105/209] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:10 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:11.365024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:11.366773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:11.376879 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:11.377584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:11.378864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:11.380496 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:11.381425 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:11.382003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:11.585576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:11.596488 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b40bd25a-7a1b-4085-b4b4-cfc61388598e tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1554 microversion: 2.1 time: 0.238507 Sep 25 00:57:11.597276 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 105/210] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:11 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1554 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:11.724652 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:11.725364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:11.731938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:11.732430 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:11.732868 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:11.733678 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:11.734212 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:11.734671 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:11.876931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:11.885432 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-11a2f192-810e-4b5b-b3dc-6b6547d5c90b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.165506 Sep 25 00:57:11.886021 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 106/211] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:11 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:12.621158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:12.621798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:12.628307 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:12.628980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:12.629363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:12.630584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:12.631215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:12.631659 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:12.787447 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:12.796972 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-39349c91-73e5-4f90-b3ff-14fe546bf592 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1554 microversion: 2.1 time: 0.179484 Sep 25 00:57:12.797602 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 106/212] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:12 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1554 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:12.909479 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:12.914831 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:12.922059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:12.922905 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:12.923700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:12.928185 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:12.928912 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:12.929391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:12.935252 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:13.134969 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:13.143200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-24e916d3-3d93-45a3-9392-f59ed818877f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.238643 Sep 25 00:57:13.143898 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 107/213] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:12 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:13.822805 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:13.823360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:13.830205 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:13.830688 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:13.831124 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:13.832031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:13.832489 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:13.832936 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:14.167682 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:14.169706 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:14.188539 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:14.189934 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:14.190655 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:14.191449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:14.191982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:14.193008 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:14.293583 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:14.308408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-de0fb5ed-e873-4495-bd90-ad55574d1734 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1554 microversion: 2.1 time: 0.490597 Sep 25 00:57:14.309228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 107/214] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:13 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1554 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:14.316038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-4dabcae0-b05b-4a4a-ab6b-710cf3f25309 req-1120cd59-6c96-43b1-963e-d9548c01f89b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "38e80e8e-ef14-4a77-b697-0d889f6cfa5f", "name": "network-vif-deleted", "tag": "9618d105-fdf7-4a07-9a29-e94ba03b6c41"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:57:14.337758 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-4dabcae0-b05b-4a4a-ab6b-710cf3f25309 req-1120cd59-6c96-43b1-963e-d9548c01f89b service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:14.338652 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-4dabcae0-b05b-4a4a-ab6b-710cf3f25309 req-1120cd59-6c96-43b1-963e-d9548c01f89b service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:14.339356 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-4dabcae0-b05b-4a4a-ab6b-710cf3f25309 req-1120cd59-6c96-43b1-963e-d9548c01f89b service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:14.392440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-4dabcae0-b05b-4a4a-ab6b-710cf3f25309 req-1120cd59-6c96-43b1-963e-d9548c01f89b service nova] Creating event network-vif-deleted:9618d105-fdf7-4a07-9a29-e94ba03b6c41 for instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f on hv-cinder-maste Sep 25 00:57:14.405302 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-4dabcae0-b05b-4a4a-ab6b-710cf3f25309 req-1120cd59-6c96-43b1-963e-d9548c01f89b service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.094025 Sep 25 00:57:14.406810 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 108/215] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:57:14 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:57:14.499934 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:14.509843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5b56e962-658f-41d4-9adf-40fcd51f292d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.346582 Sep 25 00:57:14.510467 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 108/216] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:14 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 348 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:15.335991 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:15.336692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:15.343209 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:15.343701 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:15.344122 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:15.345051 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:15.345615 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:15.346047 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:15.446924 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:15.459985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-24f79f13-1e0b-42fc-b970-14d4511aa19a tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1346 microversion: 2.1 time: 0.125432 Sep 25 00:57:15.459985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 109/217] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:15 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1346 bytes in 127 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:15.534625 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:15.535227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:15.541947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:15.542430 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:15.542951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:15.543557 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:15.544009 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:15.544428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:15.693895 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:15.701795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-253f4308-de81-4d37-a6f4-bea003d0317b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.171589 Sep 25 00:57:15.702371 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 109/218] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:15 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:16.480555 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:16.481171 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:16.488313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:16.488796 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:16.489798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:16.490797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:16.491247 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:16.491663 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:16.496279 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:16.568215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:16.577358 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-51888f54-f7a9-4dd4-988f-f4006ae98ca6 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1346 microversion: 2.1 time: 0.101272 Sep 25 00:57:16.578364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 110/219] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:16 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1346 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:16.724443 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:16.725741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:16.731845 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:16.732338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:16.732773 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:16.733615 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:16.734064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:16.734509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:16.877657 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:16.890195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-150c2395-0594-4544-88d4-89c885f28e67 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.169960 Sep 25 00:57:16.890788 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 110/220] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:16 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:17.601547 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:17.602194 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:17.609310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:17.609796 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:17.610228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:17.610916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:17.611364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:17.611812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:17.674982 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:17.685466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-01f1a270-22f1-4bc5-83e1-3895c39d7ad8 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1346 microversion: 2.1 time: 0.088627 Sep 25 00:57:17.686039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 111/221] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:17 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1346 bytes in 90 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:17.914443 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:17.915154 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:17.921559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:17.922020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:17.922451 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:17.923305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:17.923595 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:17.924010 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:18.061996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:18.071156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cd413cd7-f5c0-4b78-bfa5-d132f7a2376a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.161445 Sep 25 00:57:18.071712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 111/222] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:17 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:18.713770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:18.716167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:18.726054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:18.726738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:18.727437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:18.728334 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:18.729170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:18.729875 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:18.802642 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:18.803142 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:18.803758 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:18.804235 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lazy-loading 'fault' on Instance uuid 38e80e8e-ef14-4a77-b697-0d889f6cfa5f {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:57:18.862460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:18.873215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5e72424b-9ff6-480d-9259-eadeb812a4aa tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1351 microversion: 2.1 time: 0.166600 Sep 25 00:57:18.873828 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 112/223] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:18 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1351 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:57:19.096275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:19.096718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:19.107888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:19.108564 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:19.109043 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:19.109767 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:19.110235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:19.110668 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:19.266981 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:19.276536 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9e522b56-5c95-4612-9619-fc81ee38864f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.184224 Sep 25 00:57:19.277196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 112/224] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:19 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:19.899611 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:19.900289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:19.902699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=50 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:19.908083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:19.908603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:19.909047 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:19.909771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:19.910221 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:19.910648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:19.942825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:19.943305 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:19.943736 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:19.944175 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lazy-loading 'fault' on Instance uuid 38e80e8e-ef14-4a77-b697-0d889f6cfa5f {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:57:19.983065 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:19.992216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f0e1403a-e2fb-4761-9868-1661142fc862 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1351 microversion: 2.1 time: 0.096455 Sep 25 00:57:19.992714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 113/225] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:19 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1351 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:20.299770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:20.300375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:20.307571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:20.308050 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:20.308478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:20.309166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:20.309637 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:20.310055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:20.444014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:20.452339 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-67dd6cd6-610f-440c-8b5c-87f173558c0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.156296 Sep 25 00:57:20.452914 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 113/226] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:20 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:21.015447 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:21.016063 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:21.023113 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:21.023600 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:21.024036 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:21.024727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:21.025199 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:21.025693 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:21.055808 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:21.056280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:21.056717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:21.057160 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lazy-loading 'fault' on Instance uuid 38e80e8e-ef14-4a77-b697-0d889f6cfa5f {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:57:21.099535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:21.108316 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b09706b8-d4f1-455f-8bdb-eb8db1626cad tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1351 microversion: 2.1 time: 0.096472 Sep 25 00:57:21.108890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 114/227] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:21 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1351 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:21.475750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:21.476365 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:21.478279 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=54,nova_api:UPDATE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:21.483591 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:21.484072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:21.484503 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:21.485189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:21.485660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:21.486487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:21.619360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:21.627116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9976658c-3985-4900-914c-f5811ba1b621 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.155597 Sep 25 00:57:21.627674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 114/228] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:21 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 157 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:22.131950 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:22.132560 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:22.140129 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:22.140706 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:22.141149 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:22.141874 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:22.142329 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:22.142750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:22.177826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:22.178316 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:22.178744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:22.179182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lazy-loading 'fault' on Instance uuid 38e80e8e-ef14-4a77-b697-0d889f6cfa5f {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:57:22.223116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'f6e6706abaea4d0797af19fdb0331ead', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '5ca92ea462d24d2d8632e5c52a3dfe63', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:22.238271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-914da458-e641-4384-b0eb-b0639969582d tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 200 len: 1351 microversion: 2.1 time: 0.109688 Sep 25 00:57:22.248992 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 115/229] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:22 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 1351 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:22.651491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:22.652092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:22.658018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:22.658497 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:22.658943 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:22.659630 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:22.660078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:22.660498 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:22.799763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:22.807825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d0f8e6fa-8150-485e-87e6-ad30ac67fb99 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.160157 Sep 25 00:57:22.808390 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 115/230] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:22 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:23.262178 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:23.262691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:23.269013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:23.269516 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:23.269946 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:23.271001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:23.271460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:23.271881 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:23.285476 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] HTTP exception thrown: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Request method failure captured: Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f HTTP/1.1 Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6chTuj06AUhEKSk7yVvfE4tDyTR6q4LsRRD8XUfDTwNjtkc4J7L2oWMv_OdWvuBsfX-cynYrN4dvAZb_N_wqhg6STMKTqgQq4HcdHxDCGrspxr2dfeFqNHmV57oPVt2UmsFsq1C9-sRaB-lSbvJXYJfz_r2EgOkbD69d0zxFYSMq6U Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 5ca92ea462d24d2d8632e5c52a3dfe63 Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:57:23.290502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5ca92ea462d24d2d8632e5c52a3dfe63", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5ca92ea462d24d2d8632e5c52a3dfe63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5ca92ea462d24d2d8632e5c52a3dfe63"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5ca92ea462d24d2d8632e5c52a3dfe63"}]}] Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 5ca92ea462d24d2d8632e5c52a3dfe63 Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-DeleteServersTestJSON-1998596361-project Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: f6e6706abaea4d0797af19fdb0331ead Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-DeleteServersTestJSON-1998596361-project Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.292094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '38e80e8e-ef14-4a77-b697-0d889f6cfa5f'} Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:57:23.292938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.294341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.295710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Returning 404 to user: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:57:23.295710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e98a5fb4-f2d5-481e-b49e-12e9d099e769 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 404 len: 111 microversion: 2.1 time: 0.033637 Sep 25 00:57:23.295710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 116/231] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:23 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:57:23.415381 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:23.417665 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:23.424091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:23.428528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:23.428528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:23.428528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:23.428528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:23.429219 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:23.448622 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] HTTP exception thrown: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Request method failure captured: Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f/os-volume_attachments HTTP/1.1 Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6chTuj06AUhEKSk7yVvfE4tDyTR6q4LsRRD8XUfDTwNjtkc4J7L2oWMv_OdWvuBsfX-cynYrN4dvAZb_N_wqhg6STMKTqgQq4HcdHxDCGrspxr2dfeFqNHmV57oPVt2UmsFsq1C9-sRaB-lSbvJXYJfz_r2EgOkbD69d0zxFYSMq6U Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 5ca92ea462d24d2d8632e5c52a3dfe63 Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 00:57:23.460578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5ca92ea462d24d2d8632e5c52a3dfe63", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5ca92ea462d24d2d8632e5c52a3dfe63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5ca92ea462d24d2d8632e5c52a3dfe63"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5ca92ea462d24d2d8632e5c52a3dfe63"}]}] Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 5ca92ea462d24d2d8632e5c52a3dfe63 Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-DeleteServersTestJSON-1998596361-project Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: f6e6706abaea4d0797af19fdb0331ead Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-DeleteServersTestJSON-1998596361-project Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.462156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'server_id': '38e80e8e-ef14-4a77-b697-0d889f6cfa5f'} Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:57:23.462967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 192, in wrapper Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 303, in index Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, server_id) Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:23.464375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:23.466317 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Returning 404 to user: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:57:23.466827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c2a522e2-b8fe-444f-870b-f76f5a2a6504 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f/os-volume_attachments" status: 404 len: 111 microversion: 2.1 time: 0.055789 Sep 25 00:57:23.467458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 116/232] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 00:57:23 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f/os-volume_attachments => generated 111 bytes in 57 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:57:23.831607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:23.832133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:23.838503 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:23.838965 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:23.839397 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:23.840078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:23.840528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:23.840951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:23.980727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:23.991180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5d3adf13-0f37-4472-b51a-bf78edb37a6d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.163399 Sep 25 00:57:23.991752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 117/233] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:23 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:24.685023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:24.685661 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:24.692084 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:24.698163 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:24.698610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:24.699323 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:24.699795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:24.700401 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:24.718106 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] HTTP exception thrown: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Request method failure captured: Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: DELETE /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f HTTP/1.1 Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6chTuj06AUhEKSk7yVvfE4tDyTR6q4LsRRD8XUfDTwNjtkc4J7L2oWMv_OdWvuBsfX-cynYrN4dvAZb_N_wqhg6STMKTqgQq4HcdHxDCGrspxr2dfeFqNHmV57oPVt2UmsFsq1C9-sRaB-lSbvJXYJfz_r2EgOkbD69d0zxFYSMq6U Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 5ca92ea462d24d2d8632e5c52a3dfe63 Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 00:57:24.720388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5ca92ea462d24d2d8632e5c52a3dfe63", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5ca92ea462d24d2d8632e5c52a3dfe63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5ca92ea462d24d2d8632e5c52a3dfe63"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5ca92ea462d24d2d8632e5c52a3dfe63"}]}] Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 5ca92ea462d24d2d8632e5c52a3dfe63 Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-DeleteServersTestJSON-1998596361-project Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: f6e6706abaea4d0797af19fdb0331ead Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-DeleteServersTestJSON-1998596361-project Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.721830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '38e80e8e-ef14-4a77-b697-0d889f6cfa5f'} Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:57:24.722719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.724524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.725610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Returning 404 to user: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:57:24.725610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-66354432-2352-4c91-aafc-4f207d55edf0 tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "DELETE /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 404 len: 111 microversion: 2.1 time: 0.041245 Sep 25 00:57:24.725610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 117/234] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 00:57:24 2022] DELETE /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 111 bytes in 42 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:57:24.731365 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:24.732110 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] [instance: 38e80e8e-ef14-4a77-b697-0d889f6cfa5f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:24.741192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:24.742062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:24.742490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:24.743215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:24.743662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:24.744079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:24.760159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] HTTP exception thrown: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Request method failure captured: Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f HTTP/1.1 Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6chTuj06AUhEKSk7yVvfE4tDyTR6q4LsRRD8XUfDTwNjtkc4J7L2oWMv_OdWvuBsfX-cynYrN4dvAZb_N_wqhg6STMKTqgQq4HcdHxDCGrspxr2dfeFqNHmV57oPVt2UmsFsq1C9-sRaB-lSbvJXYJfz_r2EgOkbD69d0zxFYSMq6U Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 5ca92ea462d24d2d8632e5c52a3dfe63 Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:57:24.761802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_5ca92ea462d24d2d8632e5c52a3dfe63", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/5ca92ea462d24d2d8632e5c52a3dfe63"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5ca92ea462d24d2d8632e5c52a3dfe63"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/5ca92ea462d24d2d8632e5c52a3dfe63"}]}] Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 5ca92ea462d24d2d8632e5c52a3dfe63 Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-DeleteServersTestJSON-1998596361 Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-DeleteServersTestJSON-1998596361-project Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: f6e6706abaea4d0797af19fdb0331ead Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-DeleteServersTestJSON-1998596361-project Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.763264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '38e80e8e-ef14-4a77-b697-0d889f6cfa5f'} Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:57:24.764607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. Sep 25 00:57:24.765899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:57:24.767265 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] Returning 404 to user: Instance 38e80e8e-ef14-4a77-b697-0d889f6cfa5f could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:57:24.767265 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-21584bce-12c7-482e-8fc5-10d1cf3ab3de tempest-DeleteServersTestJSON-1998596361 tempest-DeleteServersTestJSON-1998596361-project] 10.222.0.52 "GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f" status: 404 len: 111 microversion: 2.1 time: 0.035852 Sep 25 00:57:24.767265 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 118/235] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:24 2022] GET /compute/v2.1/servers/38e80e8e-ef14-4a77-b697-0d889f6cfa5f => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:57:25.016809 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:25.017755 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:25.025719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:25.026557 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:25.027144 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:25.037321 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:25.038564 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.010s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:25.039674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:25.275345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:25.287594 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1637b968-430a-4f7a-b8fe-a5de4ef55062 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.276008 Sep 25 00:57:25.303527 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 118/236] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:25 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:26.314469 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:26.315083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:26.321947 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:26.322615 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:26.323415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:26.324426 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:26.325291 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:26.325749 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:26.540954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:26.550251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8aa77925-457f-414f-9e26-8422ac980dbb tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.239601 Sep 25 00:57:26.551990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 119/237] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:26 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:27.575741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:27.577321 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:27.584948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:27.585458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:27.586050 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:27.586970 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:27.587399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:27.592225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:28.106987 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:28.133593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5aee0d1b-c74e-4907-a528-4bdb52f206f5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.561477 Sep 25 00:57:28.136137 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 119/238] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:27 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 566 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:29.160266 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:29.162193 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:29.169589 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:29.170340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:29.170968 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:29.171905 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:29.172670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:29.173601 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:29.953907 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:29.976451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f8c6892f-ed6c-443f-b5e4-bbe42344ab31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.822758 Sep 25 00:57:29.988306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 120/239] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:29 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 835 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:31.003447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:31.004931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:31.014795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:31.015620 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:31.016820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:31.020312 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:31.020815 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:31.021272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:31.344528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:31.364986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6b618876-cebc-4b42-a231-24dba4703420 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.361705 Sep 25 00:57:31.365819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 120/240] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:30 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:32.400132 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:32.401552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:32.411851 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:32.412764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:32.413494 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:32.414601 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:32.415439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:32.416083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:32.939215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:32.952197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-975b9545-0fb9-4882-8c7c-e30e5fa958e2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1521 microversion: 2.1 time: 0.560544 Sep 25 00:57:32.953570 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 121/241] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:32 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1521 bytes in 567 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:33.981288 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:33.981973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:33.991726 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:33.992249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:33.994038 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:33.994816 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:33.995330 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:33.997274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:34.197836 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:34.223497 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9732dda7-9a66-4e44-91e4-407d170ecf8c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1501 microversion: 2.1 time: 0.248679 Sep 25 00:57:34.224169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 121/242] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:33 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1501 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:34.249833 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:34.251047 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:34.267745 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:34.272382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:34.273022 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:34.274130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:34.274810 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:34.275436 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:34.531302 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:34.542448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-35af96f1-b505-4e80-b4ca-b91b36e433cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1516 microversion: 2.1 time: 0.296337 Sep 25 00:57:34.544931 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 122/243] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:34 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1516 bytes in 299 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:39.489391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:57:39.492942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:39.501679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:39.501941 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:39.505391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:39.506766 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:39.508761 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:39.512658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:39.554872 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lazy-loading 'flavor' on Instance uuid 3c9017f3-8bdb-4c13-9c50-646241c8f1ba {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:57:39.674634 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-96e2a64f-c3ff-43b0-84bf-2484ac9b453d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba/action" status: 200 len: 32 microversion: 2.1 time: 0.189774 Sep 25 00:57:39.675254 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 122/244] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 00:57:39 2022] POST /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba/action => generated 32 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 25 00:57:39.687203 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:39.687814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:39.696663 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:39.697409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:39.698024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:39.698971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:39.699756 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:39.700371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:39.974522 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:39.986017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d8ceb4c5-850a-41a7-81ab-e17c3fc6a269 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.303262 Sep 25 00:57:40.003103 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 123/245] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:39 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 322 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:40.039375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3d11385c-14c0-4736-af26-42ed48b39655 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Action: 'create', calling method: .version_select at 0x7fe22da664c0>, body: {"keypair": {"name": "tempest-TestVolumeBootPattern-1527375203", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBMry85yk7ju5OMioDfw/VhTnugjFqdgfSZ5PFj2dghtkpKw/lDAy7Kis0Z3qKXqA23mzXmatnS2gshYjgivnYobyHTsm4i0ug5x5A0fYEOV+77LzdKjVOf7tGIlov+uSWg=="}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:57:40.062558 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-3d11385c-14c0-4736-af26-42ed48b39655 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Getting quotas for project 46f19fd9ad1e4880961d1eee75883687. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:57:40.068372 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-3d11385c-14c0-4736-af26-42ed48b39655 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Getting quotas for user c19b10ebc31244dbb2b1539d94ac281d and project 46f19fd9ad1e4880961d1eee75883687. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:57:40.116915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3d11385c-14c0-4736-af26-42ed48b39655 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 400 microversion: 2.1 time: 0.407745 Sep 25 00:57:40.118285 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 123/246] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 00:57:39 2022] POST /compute/v2.1/os-keypairs => generated 400 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:57:41.032162 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:41.036694 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:41.046052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:41.046650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:41.047183 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:41.047984 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:41.048781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:41.049477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:41.257686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:41.267138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4d37b53d-e0e6-404b-859f-4ba3323d57d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.245500 Sep 25 00:57:41.267719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 124/247] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:41 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:42.290359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:42.291093 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:42.297974 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:42.298462 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:42.298894 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:42.299593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:42.300063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:42.301055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:42.306294 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:42.440697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:42.449584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9ef5a42b-f335-4e77-9f65-9f63ce85dd4e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.163056 Sep 25 00:57:42.450157 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 124/248] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:42 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:43.472588 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:43.473323 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:43.480276 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:43.480690 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:43.481097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:43.481833 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:43.482282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:43.482699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:43.626194 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:43.634789 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-43cd2950-8736-4781-8810-dd60ab7d9c77 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.166512 Sep 25 00:57:43.635361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 125/249] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:43 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:44.658169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:44.658776 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:44.665690 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:44.666169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:44.666603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:44.667282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:44.667730 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:44.668148 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:44.811475 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:44.814242 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e138f390-23a9-490f-b68d-9716d2b4d7fd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.159864 Sep 25 00:57:44.814786 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 125/250] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:44 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:45.836906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:45.837613 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:45.844025 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:45.844629 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:45.845064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:45.845778 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:45.846229 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:45.846646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:45.995281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:45.998748 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:46.003612 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-493e75a3-7c49-4183-9240-d648389ce74f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.170981 Sep 25 00:57:46.004174 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 126/251] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:45 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:47.026367 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:47.027004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:47.035911 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:47.036607 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:47.037042 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:47.037895 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:47.038595 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:47.038785 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:47.172995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:47.181616 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-04bb203d-7b53-44dc-afaf-0cd05865fdca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.159010 Sep 25 00:57:47.182179 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 126/252] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:47 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:48.204740 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:48.205331 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:48.212035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:48.212514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:48.212944 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:48.213656 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:48.214099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:48.214516 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:48.354938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:48.364410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5ef88740-8798-4188-829a-6ca88e052523 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.163470 Sep 25 00:57:48.364972 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 127/253] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:48 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:49.387060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:49.387852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:49.393769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:49.394845 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:49.395361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:49.395994 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:49.396460 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:49.396888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:49.533727 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:49.541759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-13537e69-79ca-42a1-b30e-5d9d97f20038 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.158540 Sep 25 00:57:49.542353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 127/254] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:49 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:50.565003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:50.565630 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:50.568338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=36 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:50.573864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:50.574348 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:50.574910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:50.575635 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:50.576082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:50.576500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:50.713798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:50.721377 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ebb6746c-78a7-400b-8a68-12d87d905546 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.160216 Sep 25 00:57:50.721944 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 128/255] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:50 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:51.746937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:51.747598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:51.749639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=4 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:51.750344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=37,nova_api:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:57:51.756955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:51.757624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:51.758101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:51.758806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:51.759274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:51.759693 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:51.902364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:51.910787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8c69488c-3863-4e25-a510-a59b281762b3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.167851 Sep 25 00:57:51.911377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 128/256] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:51 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:52.932653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:52.933282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:52.939676 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:52.940157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:52.940594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:52.941846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:52.942296 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:52.942721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:53.070151 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:53.077975 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6ecb087a-3bc2-4524-b0ad-d4509b2391ff tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.149042 Sep 25 00:57:53.078540 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 129/257] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:52 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:54.102228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:54.102827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:54.108923 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:54.109456 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:54.110010 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:54.110656 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:54.111189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:54.111720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:54.252404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:54.260463 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-dc543e10-f7a8-4b9e-b6f6-17273b95fff1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.163058 Sep 25 00:57:54.261024 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 129/258] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:54 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:55.285682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:55.286687 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:55.299148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:55.299642 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:55.300076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:55.302085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:55.302573 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:55.303053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:55.441982 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:55.449778 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9459a2e0-81b7-4733-a5c5-b18006c1d79d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.169681 Sep 25 00:57:55.450344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 130/259] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:55 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:56.473350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:56.474237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:56.480503 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:56.480982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:56.481480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:56.482173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:56.482626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:56.483053 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:56.622398 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:56.631114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9f131660-08f3-483e-b204-539d8d49e75a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.162189 Sep 25 00:57:56.631685 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 130/260] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:56 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:56.931852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:56.932348 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:56.938483 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:56.938963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:56.939396 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:56.940089 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:56.940547 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:56.941679 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:56.967578 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-895df1d7-e392-4489-ad5a-793c11f398e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/ips" status: 200 len: 148 microversion: 2.1 time: 0.039269 Sep 25 00:57:56.968077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 131/261] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 00:57:56 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/ips => generated 148 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:57:57.653807 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:57.654448 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:57.660358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:57.660847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:57.661296 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:57.661985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:57.662461 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:57.663286 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:57.795067 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:57.803857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-10b52a2d-5187-49a9-be93-aa6ea74e4afe tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.154050 Sep 25 00:57:57.804444 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 131/262] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:57 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:57:58.452212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "28ae3ae1-26a0-462e-8a8f-d89670d28529"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:57:58.455212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:58.462407 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:58.462889 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:58.463357 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:58.464027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:58.464483 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:58.464899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:58.768214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6fcb857a-db29-4740-be2d-45afa586760e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.319499 Sep 25 00:57:58.771819 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 132/263] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 00:57:58 2022] POST /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments => generated 194 bytes in 324 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:57:58.827116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:57:58.827708 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:57:58.835345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:58.835832 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:58.836265 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:58.836969 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:57:58.837618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:57:58.838198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:57:58.984753 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:57:58.994999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-54563eee-54b4-490d-acf5-c19121918d19 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.171611 Sep 25 00:57:58.995574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 132/264] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:57:58 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:00.022080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:00.022955 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:00.029399 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:00.029888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:00.030537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:00.031238 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:00.031776 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:00.032204 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:00.175850 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:00.184093 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-da54f5f2-1a08-4a5b-82eb-424289bc1434 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.167810 Sep 25 00:58:00.184676 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 133/265] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:00 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:01.208303 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:01.209201 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:01.215481 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:01.216196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:01.216621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:01.217930 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:01.218960 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:01.219893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:01.473451 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:01.483149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-009a5990-93ae-40fc-a425-1ad24a800c96 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.278925 Sep 25 00:58:01.490209 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 133/266] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:01 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:02.519506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:02.521621 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:02.531583 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:02.533529 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:02.535128 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:02.537043 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:02.538680 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:02.539802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:02.873887 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:02.883984 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5ddd4281-5937-4fd0-bc01-44e3ab39bfd2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.370707 Sep 25 00:58:02.884872 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 134/267] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:02 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 381 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:03.907718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:03.909821 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:03.918753 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:03.920120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:03.920835 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:03.922659 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:03.923364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:03.924063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.234858 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:04.266609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7c4fd219-1755-4325-9ca0-d1a57607a2ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1502 microversion: 2.1 time: 0.362797 Sep 25 00:58:04.267514 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 134/268] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:03 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1502 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:04.294552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "53ea15f4-27cc-43e7-9175-2745549a16ee"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:04.297045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:04.306595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:04.307101 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:04.307538 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.308250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:04.308704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:04.309133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.335313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] HTTP exception thrown: Cannot 'attach_volume' instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba while it is in vm_state rescued Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Request method failure captured: Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: POST /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba/os-volume_attachments HTTP/1.1 Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 74 Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cb8QFEWahSITkrpCBqxYYBD9zi_oLSAVWAk9whdwsp48CleshPfYnV97ECBsJa63kh6to7hObTL2AlKDs6Ggck8j1eD8_mIGQLRF0aMsS9KlL5zrlzoseQDGerOSLOpzlzHV2LyCeRa0pLQWwIucZcMEZw7qNiXQLyFloWXCSSLWQ Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 28568584195d475fbc52d76185e09044 Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:58:04.337290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_28568584195d475fbc52d76185e09044", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/28568584195d475fbc52d76185e09044"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}] Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 28568584195d475fbc52d76185e09044 Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 8eabe2a74a604c47bc2e5c88b960dcb6 Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {"volumeAttachment": {"volumeId": "53ea15f4-27cc-43e7-9175-2745549a16ee"}} Sep 25 00:58:04.339053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Cannot 'attach_volume' instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba while it is in vm_state rescued Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'server_id': '3c9017f3-8bdb-4c13-9c50-646241c8f1ba', 'body': {'volumeAttachment': {'volumeId': '53ea15f4-27cc-43e7-9175-2745549a16ee'}}} Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba in vm_state rescued. Cannot attach_volume while the instance is in this state. Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:58:04.340134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 398, in create Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'attach_volume' instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba while it is in vm_state rescued Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:58:04.341704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:58:04.343415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:58:04.343415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:58:04.343415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'attach_volume' instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba while it is in vm_state rescued Sep 25 00:58:04.343415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:58:04.343415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Returning 409 to user: Cannot 'attach_volume' instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba while it is in vm_state rescued {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:58:04.343415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-775ce5cd-2da0-45ab-bb01-9865cd6199ca tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba/os-volume_attachments" status: 409 len: 152 microversion: 2.1 time: 0.050097 Sep 25 00:58:04.343415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 135/269] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 00:58:04 2022] POST /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba/os-volume_attachments => generated 152 bytes in 52 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 25 00:58:04.349565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:04.350105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:04.356298 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:04.356820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:04.357303 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.358058 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:04.358519 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:04.363310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.445017 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cdc1e849-d6c5-4333-8f41-d08a7786cf32 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba/action" status: 202 len: 0 microversion: 2.1 time: 0.099380 Sep 25 00:58:04.445458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 135/270] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 00:58:04 2022] POST /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba/action => generated 0 bytes in 102 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 00:58:04.455437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:04.456047 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:04.473440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:04.474771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:04.475633 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.478641 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:04.479550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:04.481115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.698585 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:04.709250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8f01a64d-0499-47ae-9871-d91491dd60cc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1510 microversion: 2.1 time: 0.259185 Sep 25 00:58:04.710155 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 136/271] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:04 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1510 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:04.795577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:04.796504 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:04.807225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:04.808915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:04.809664 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.810651 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:04.811323 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:04.812720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:04.848357 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-058f1d9a-591f-4fd9-a89c-7a05cbe67a53 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/ips" status: 200 len: 153 microversion: 2.20 time: 0.057376 Sep 25 00:58:04.856311 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 136/272] 10.222.0.52 () {60 vars in 1289 bytes} [Sun Sep 25 00:58:04 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/ips => generated 153 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 00:58:05.734836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:05.735367 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:05.762222 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:05.763438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:05.766971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:05.770312 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:05.771992 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:05.772621 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:06.006448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:06.017892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-64c09cd9-a477-48a9-8a60-7b65e5997f31 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1510 microversion: 2.1 time: 0.287010 Sep 25 00:58:06.018472 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 137/273] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:05 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1510 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:06.545846 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-af52faca-bf18-417f-addf-fd5307f99d0f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: .version_select at 0x7fe22daba310>, body: {"keypair": {"name": "tempest-keypair-284669568", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBPICnRhdB8kMQnnjtedRj+fwbfr7OmxbH0IclbhWCF3sblQGi9rZT6eu8/LL4D2JcuQ8nnPXXzDp1Tm3CmU9YrkQbJtnY69t0IHLj5isDOw6eRuR3vYG5E6MnZYXg4N3ZQ=="}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:06.560921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-af52faca-bf18-417f-addf-fd5307f99d0f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:58:06.568109 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-af52faca-bf18-417f-addf-fd5307f99d0f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for user 96386a16f0b54b7c94d32bd948caf118 and project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:58:06.610506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-af52faca-bf18-417f-addf-fd5307f99d0f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.070925 Sep 25 00:58:06.615204 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 137/274] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 00:58:06 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 77 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:58:07.041807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:07.042446 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:07.050059 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:07.050557 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:07.050996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:07.051702 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:07.052158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:07.052581 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:07.278390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:07.288692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a1febfd4-74a6-4103-888f-d196a53d22d9 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1510 microversion: 2.1 time: 0.250933 Sep 25 00:58:07.290972 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 138/275] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:07 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1510 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:08.314325 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:08.315807 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:08.324813 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:08.325758 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:08.326472 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:08.327439 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:08.328282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:08.329268 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:08.502534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:08.503223 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:08.516370 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:08.516947 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:08.517424 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:08.518146 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:08.529965 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:08.530763 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:08.590070 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lazy-loading 'flavor' on Instance uuid 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:58:08.708020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:08.718059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ad5ddd06-b2a8-4b38-8dca-0e7225dc8c21 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1510 microversion: 2.1 time: 0.407781 Sep 25 00:58:08.719213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 138/276] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:08 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1510 bytes in 410 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:08.914449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1371672477", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5dc49b54-686c-4619-ad4c-65c72c26fd31"}], "security_groups": [{"name": "tempest-securitygroup--203677135"}], "key_name": "tempest-keypair-284669568", "user_data": "}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:09.133646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d3e276e7-3bad-4f9a-8a6a-9e63b00553c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/action" status: 202 len: 0 microversion: 2.20 time: 0.633823 Sep 25 00:58:09.134434 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 139/277] 10.222.0.52 () {62 vars in 1319 bytes} [Sun Sep 25 00:58:08 2022] POST /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/action => generated 0 bytes in 636 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 25 00:58:09.150422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:09.151189 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:09.164280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:09.165389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:09.166170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:09.167106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:09.167896 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:09.168673 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:09.276908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] validate_networks() for [('5dc49b54-686c-4619-ad4c-65c72c26fd31', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:58:09.470224 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:09.474451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=4,nova_cell1:INSERT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:09.483796 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:09.490045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:09.492468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a3fc243b-3f7e-4d69-a4a3-eeaeffccb777 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.351170 Sep 25 00:58:09.496226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 140/278] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:09 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:09.742890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:09.744545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:09.756920 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:09.762823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:09.763313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:09.765293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:09.766922 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:09.772165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:09.926272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:58:09.929235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:58:09.930379 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:58:09.931918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:58:09.932537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:58:09.948255 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:09.949013 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:09.951894 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:09.954683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:09.956980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:09.957780 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:09.991053 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:58:09.997101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for user 96386a16f0b54b7c94d32bd948caf118 and project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:58:10.006818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:10.016146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:58:10.019542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-36f16b96-30c4-499a-a75a-0dfdc14a412f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1516 microversion: 2.1 time: 0.278520 Sep 25 00:58:10.020334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:58:10.021645 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 141/279] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:09 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1516 bytes in 283 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:10.207885 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-78ddf2fd-2a49-4675-9ada-0cac28fba38a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.301988 Sep 25 00:58:10.210028 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 139/280] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:58:08 2022] POST /compute/v2.1/servers => generated 401 bytes in 1306 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 00:58:10.222263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1c920f29-3966-467a-afbe-5813057b52b2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:10.223343 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1c920f29-3966-467a-afbe-5813057b52b2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:10.323840 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-78157052", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1527375203", "security_groups": [{"name": "tempest-secgroup-smoke-1834758444"}], "block_device_mapping_v2": [{"uuid": "b08083ef-2e11-4ed7-87d3-6520c8d3cf21", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "31fa7821-6345-4f48-b070-9ceab6d12adf"}]}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:10.553115 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] validate_networks() for [('31fa7821-6345-4f48-b070-9ceab6d12adf', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:58:10.960907 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1c920f29-3966-467a-afbe-5813057b52b2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:10.972025 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1c920f29-3966-467a-afbe-5813057b52b2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1241 microversion: 2.1 time: 0.751390 Sep 25 00:58:10.989131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 142/281] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:10 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1241 bytes in 757 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:10.989131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:10.989131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:10.994813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:10.995310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:10.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:10.996438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:10.997052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:10.997507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:11.227203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:58:11.228291 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:58:11.229561 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:58:11.230135 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:58:11.230706 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:58:11.243237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:11.243727 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:11.244463 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:11.244862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:11.245508 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:11.246211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:11.282364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Getting quotas for project 46f19fd9ad1e4880961d1eee75883687. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:58:11.289957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Getting quotas for user c19b10ebc31244dbb2b1539d94ac281d and project 46f19fd9ad1e4880961d1eee75883687. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:58:11.305898 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:11.317116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:11.320396 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:11.321850 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8888977f-18fb-4a63-b1c1-4c316d678a44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.347338 Sep 25 00:58:11.338906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 143/282] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:10 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:11.388419 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:58:11.395310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b08083ef-2e11-4ed7-87d3-6520c8d3cf21',volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:58:11.623071 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ec473234-818b-476d-959f-aaaa29ef7cde tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.306154 Sep 25 00:58:11.624477 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 140/283] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:58:10 2022] POST /compute/v2.1/servers => generated 406 bytes in 1309 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 00:58:11.635326 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-90480acf-4522-47fa-b869-782969e9dadf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:11.635948 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-90480acf-4522-47fa-b869-782969e9dadf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:11.721105 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-90480acf-4522-47fa-b869-782969e9dadf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:11.749318 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-90480acf-4522-47fa-b869-782969e9dadf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1093 microversion: 2.1 time: 0.110095 Sep 25 00:58:11.752513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 144/284] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:11 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1093 bytes in 122 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:11.996004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:11.996858 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:12.013048 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:12.014013 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:12.014561 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:12.015465 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:12.016324 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:12.017090 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:12.149418 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:12.159064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-74fddb30-ceec-420c-85db-01d05dca5e73 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1245 microversion: 2.1 time: 0.166837 Sep 25 00:58:12.160483 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 141/285] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:11 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1245 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:12.353359 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:12.354246 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:12.365838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:12.366464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:12.366990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:12.369697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:12.370860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:12.371301 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:12.776407 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:12.778143 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:12.785674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:12.789375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:12.790158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:12.791339 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:12.792023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:12.792665 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:12.797624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:12.811443 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:12.827799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:12.829321 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:12.830553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-dc052a9b-1490-4dae-bae5-aa16ad52849b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.484496 Sep 25 00:58:12.831822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 145/286] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:12 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:12.930567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f5536e74-e3d9-4d89-a030-0872a07ec813 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'create', calling method: .version_select at 0x7fe22db9d1f0>, body: {"keypair": {"name": "tempest-keypair-1456756188", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBCEDuYPjGOfDlhaEv9IJXoxx9GRalA4VYUjkyZGzN/A+Iv6ifbb6YdKPsFh/hibp4McCuQJUOhx2dv6YXmyxgGHYLLazN6AkXmqrJt9tb3tg/g9z2CnNIq82gifyuxHwBA=="}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:12.952088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-f5536e74-e3d9-4d89-a030-0872a07ec813 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Getting quotas for project 28568584195d475fbc52d76185e09044. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:58:12.957462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-f5536e74-e3d9-4d89-a030-0872a07ec813 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Getting quotas for user 8eabe2a74a604c47bc2e5c88b960dcb6 and project 28568584195d475fbc52d76185e09044. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:58:12.984452 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f5536e74-e3d9-4d89-a030-0872a07ec813 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.057820 Sep 25 00:58:12.993441 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 146/287] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 00:58:12 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:58:13.020984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:13.029365 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-64cfa5d9-33a3-4409-8ec8-c4fd0b4dd7e7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1135 microversion: 2.1 time: 0.258768 Sep 25 00:58:13.045298 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 142/288] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:12 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1135 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:13.183492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:13.184231 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:13.193131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:13.193749 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:13.194258 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:13.195072 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:13.195529 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:13.195958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:13.324424 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:13.336724 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2dc6bd2d-464c-495e-aa91-cd5c52a2eb7e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1293 microversion: 2.1 time: 0.157546 Sep 25 00:58:13.339289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 147/289] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:13 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1293 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:13.859406 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:13.859901 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:13.866639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:13.867440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:13.868123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:13.869051 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:13.869751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:13.870393 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:14.055888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:14.056591 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:14.064124 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:14.064776 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:14.066239 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:14.067263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:14.067797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:14.068293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:14.166446 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:14.180032 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:14.185677 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:14.188499 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-791890f5-4057-485d-ad40-07e069432642 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.333078 Sep 25 00:58:14.189358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 143/290] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:13 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 335 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:14.246937 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:14.256762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3afb9996-525d-4104-b414-5c97dfadf926 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1135 microversion: 2.1 time: 0.208638 Sep 25 00:58:14.262599 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 148/291] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:14 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1135 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:14.370622 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:14.371255 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:14.379539 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:14.380200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:14.380683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:14.381497 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:14.381961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:14.382387 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:14.508578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:14.517070 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-401a9439-3b8d-4e44-a89d-cf55749527ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1293 microversion: 2.1 time: 0.154514 Sep 25 00:58:14.518814 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 144/292] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:14 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1293 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:15.105274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-ServerRescueNegativeTestJSON-server-2082726982", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "e04e78f2-fc42-4b3e-9ba5-62cef58448f8"}], "security_groups": [{"name": "tempest-securitygroup--128480623"}], "key_name": "tempest-keypair-1456756188", "user_data": "}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:15.216014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:15.217280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:15.225451 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:15.226119 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:15.234870 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] validate_networks() for [('e04e78f2-fc42-4b3e-9ba5-62cef58448f8', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:58:15.243078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:15.244211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:15.254609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:15.255083 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:15.448151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:15.467044 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:15.470632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:15.474413 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d841ec00-7020-4b2b-ba60-78fecf7caebe tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.261153 Sep 25 00:58:15.476399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 145/293] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:15 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 265 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:58:15.513462 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:15.514806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:15.526166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:15.527166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:15.528810 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:15.532421 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:15.536414 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:15.537317 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:15.760963 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:15.770603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e833d017-7fd1-4464-8bd0-901f71255f4b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1135 microversion: 2.1 time: 0.278296 Sep 25 00:58:15.771254 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 146/294] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:15 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1135 bytes in 294 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:15.787304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:15.787924 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:15.801475 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:15.807861 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:15.818035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.012s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:15.819061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:15.819715 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:15.820332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:15.921902 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:15.940012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0ccf22af-b42c-487f-8465-de2c49eb8a0e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1293 microversion: 2.1 time: 0.157482 Sep 25 00:58:15.949959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 147/295] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:15 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1293 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:16.501174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:16.507555 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:16.515465 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:16.527411 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.012s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:16.530958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:16.535621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:16.539602 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:16.543116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:16.691094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:58:16.691581 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:58:16.691993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:58:16.692409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:58:16.692889 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:58:16.712858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:16.717283 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:16.722544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:16.723614 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:16.724247 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:16.725057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:16.751191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Getting quotas for project 28568584195d475fbc52d76185e09044. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:58:16.758192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Getting quotas for user 8eabe2a74a604c47bc2e5c88b960dcb6 and project 28568584195d475fbc52d76185e09044. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:58:16.783188 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:58:16.788091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:58:16.893818 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:16.915732 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:16.917623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:16.919159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c39285c2-2597-4c88-97d7-4b400be37591 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.421878 Sep 25 00:58:16.920298 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 148/296] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:16 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:16.925669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:16.927873 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-dc21f340-77c8-47c5-98a3-a75ed3f2e1ae tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 408 microversion: 2.1 time: 1.826840 Sep 25 00:58:16.935656 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:16.936141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 149/297] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:58:15 2022] POST /compute/v2.1/servers => generated 408 bytes in 1836 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 00:58:16.951061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:16.951802 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:16.952387 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:16.953468 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:16.953905 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bcccc6e7-e1b3-4d7d-9108-91cbdc48fc97 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:16.954892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-bcccc6e7-e1b3-4d7d-9108-91cbdc48fc97 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:16.955441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:16.956972 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:17.077485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-bcccc6e7-e1b3-4d7d-9108-91cbdc48fc97 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:17.085879 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bcccc6e7-e1b3-4d7d-9108-91cbdc48fc97 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1246 microversion: 2.1 time: 0.148436 Sep 25 00:58:17.086661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 150/298] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:16 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1246 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:17.094209 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:17.094822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:17.113538 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:17.119991 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:17.121762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:17.122407 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:17.123853 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fba47afa-6db3-4de2-b599-f34ab9c8e871 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1191 microversion: 2.1 time: 0.201953 Sep 25 00:58:17.136330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:17.136330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:17.136330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:17.141979 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 149/299] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:16 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1191 bytes in 221 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:17.427174 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:17.440764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c5c4fe46-cc13-4f85-8f7e-1d5720df0a50 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1299 microversion: 2.1 time: 0.350751 Sep 25 00:58:17.441779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 151/300] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:17 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1299 bytes in 355 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:17.949806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:17.950594 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:17.964442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:17.964959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:17.965429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:17.967310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:17.967859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:17.968232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:18.108938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:18.108938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:18.125099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:18.137772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:18.138306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:18.139067 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:18.139536 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:18.141040 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:18.244904 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:18.263314 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:18.264860 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:18.266090 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1ba19590-8db8-4115-bdba-9c8af6a20234 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.321767 Sep 25 00:58:18.271364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 150/301] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:17 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 328 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:18.276234 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:18.277568 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:18.311030 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:18.311554 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:18.311991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:18.312709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:18.313167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:18.313717 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:18.375180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:18.400432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bf752d67-8264-4a2d-b382-0ec811651a58 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1242 microversion: 2.1 time: 0.296247 Sep 25 00:58:18.411416 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 152/302] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:18 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1242 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:18.469462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:18.470085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:18.475918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:18.492056 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:18.494107 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:18.494716 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:18.498592 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1a187b86-330c-491d-a5c2-86c662cd2107 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1209 microversion: 2.1 time: 0.220839 Sep 25 00:58:18.502098 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:18.503374 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:18.503764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:18.507359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 151/303] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:18 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1209 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:18.713797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:18.730357 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-fb5ebf4e-04b1-42e3-a6e5-d0a370e8d6fe tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1299 microversion: 2.1 time: 0.266326 Sep 25 00:58:18.733301 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 153/304] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:18 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1299 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:19.301869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:19.302586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:19.317496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:19.321148 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:19.321903 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:19.323509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:19.325740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:19.327282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:19.431355 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:19.431978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:19.438743 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:19.439556 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:19.440895 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:19.441833 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:19.442454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:19.443262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:19.659784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:19.682739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-966483e0-a0f8-43c5-9df6-515eabbde248 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1306 microversion: 2.1 time: 0.255095 Sep 25 00:58:19.712463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 154/305] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:19 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1306 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:19.734670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:19.735482 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:19.748654 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:19.754492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:19.760381 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:19.761560 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:19.762925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:19.763535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:19.870648 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:19.895491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:19.901061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:19.906743 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4ee27f10-73b6-4be5-b85e-3fc2fb4dcea1 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.610451 Sep 25 00:58:19.909376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 152/306] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:19 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 616 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:19.923491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:19.924144 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:19.949975 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:19.953392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:19.953961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:19.960504 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:19.962784 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:19.964188 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:20.225219 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:20.238045 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-817c01c3-691d-4573-af4d-570ffaa26ff4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.324562 Sep 25 00:58:20.248824 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 153/307] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:19 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:20.479545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:20.495928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0a150225-a65d-4b8c-b6b2-53deec20165a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.780447 Sep 25 00:58:20.500217 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 155/308] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:19 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:20.733697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:20.736034 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:20.751775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:20.755848 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:20.758060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:20.759266 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:20.761424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:20.762445 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:20.948545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:20.949216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:20.955209 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=84,nova_api:INSERT=4 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:20.967439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:20.968491 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:20.969159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:20.974577 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:20.975572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:20.976382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:21.073638 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:21.096904 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-966a5e20-127c-4d56-b3a4-1d486ba7ba4d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1304 microversion: 2.1 time: 0.383648 Sep 25 00:58:21.098109 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 154/309] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:20 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1304 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:21.269221 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:21.270211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:21.282332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=4 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:21.285110 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:21.286310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:21.288609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:21.290770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:21.291970 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:21.293801 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:21.304354 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=16 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:21.627825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:21.641605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:21.643099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:21.644495 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b8b5d721-5247-4848-9042-f3552ea9e948 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.705042 Sep 25 00:58:21.645603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 156/310] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:20 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 707 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:21.658238 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-0375ccdd-879c-4aec-a017-200862346ad7 req-4e0528bc-3d8d-4116-b813-ade13fbb11c4 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6892e391-59db-43ba-8843-ce34ade53390", "tag": "36da5046-389c-4afa-9d14-41833f5b0db3"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:21.687713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-0375ccdd-879c-4aec-a017-200862346ad7 req-4e0528bc-3d8d-4116-b813-ade13fbb11c4 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:21.690346 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-0375ccdd-879c-4aec-a017-200862346ad7 req-4e0528bc-3d8d-4116-b813-ade13fbb11c4 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:21.691122 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-0375ccdd-879c-4aec-a017-200862346ad7 req-4e0528bc-3d8d-4116-b813-ade13fbb11c4 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:21.700026 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:21.726189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b3aa8373-9a50-446d-97a5-179d7f4ec6f6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.462097 Sep 25 00:58:21.729336 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-0375ccdd-879c-4aec-a017-200862346ad7 req-4e0528bc-3d8d-4116-b813-ade13fbb11c4 service nova] Creating event network-changed:36da5046-389c-4afa-9d14-41833f5b0db3 for instance 6892e391-59db-43ba-8843-ce34ade53390 on hv-cinder-maste Sep 25 00:58:21.730586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 155/311] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:21 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 467 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:21.747513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-0375ccdd-879c-4aec-a017-200862346ad7 req-4e0528bc-3d8d-4116-b813-ade13fbb11c4 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.100490 Sep 25 00:58:21.748195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:21.749236 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 157/312] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:58:21.751333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:21.772448 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=91,nova_api:INSERT=7 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:21.778844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:21.779500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:21.779961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:21.780694 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:21.783190 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:21.784680 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:22.123200 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:22.124207 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:22.137288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:22.137807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:22.138254 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:22.138977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:22.139433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:22.150583 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:22.331542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:22.341190 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-605af305-ab74-425e-8f18-9775bbf63edc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.609239 Sep 25 00:58:22.346645 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 156/313] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:21 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:22.650927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:22.667140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e665546a-959c-4893-9961-3dd755b8709f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.548174 Sep 25 00:58:22.673091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:22.673920 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:22.680097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 158/314] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:22 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 562 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:22.688483 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:22.688985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:22.689506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:22.690327 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:22.691033 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:22.691453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:22.761445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:22.763099 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:22.782589 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:22.785457 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:22.786059 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:22.786813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:22.790165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:22.790627 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:23.092539 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:23.103773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0b21c709-f8fb-46bc-9056-5eff029f1d46 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.348030 Sep 25 00:58:23.104952 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 159/315] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:22 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 350 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:23.378230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:23.378993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:23.380025 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:23.396337 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:23.397805 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:23.398976 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-378d8c8b-2671-4aaa-8fc2-84141b4645eb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.730221 Sep 25 00:58:23.401333 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:23.401919 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:23.402044 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 157/316] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:22 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 734 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:23.402485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:23.403443 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:23.404128 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:23.404750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:23.617514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:23.627150 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8bc878fb-6cf1-438e-9502-8e121e483f24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.253680 Sep 25 00:58:23.628674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 160/317] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:23 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:23.698431 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:23.699011 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:23.707294 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:23.708918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:23.709915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:23.710869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:23.711530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:23.712183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:23.717557 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-118c4c1a-5c99-4c7c-a713-7fe6c629415f req-83b69077-bd84-47d0-9fb5-58e345343fea service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "621dfc45-048c-452f-86ca-ffdaeeb5f919", "tag": "a554103e-40f9-42b8-b04a-4fc7d8888240"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:23.732810 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-118c4c1a-5c99-4c7c-a713-7fe6c629415f req-83b69077-bd84-47d0-9fb5-58e345343fea service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:23.735291 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-118c4c1a-5c99-4c7c-a713-7fe6c629415f req-83b69077-bd84-47d0-9fb5-58e345343fea service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:23.735878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-118c4c1a-5c99-4c7c-a713-7fe6c629415f req-83b69077-bd84-47d0-9fb5-58e345343fea service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:23.754311 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-118c4c1a-5c99-4c7c-a713-7fe6c629415f req-83b69077-bd84-47d0-9fb5-58e345343fea service nova] Creating event network-changed:a554103e-40f9-42b8-b04a-4fc7d8888240 for instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 on hv-cinder-maste Sep 25 00:58:23.759496 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-118c4c1a-5c99-4c7c-a713-7fe6c629415f req-83b69077-bd84-47d0-9fb5-58e345343fea service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.046630 Sep 25 00:58:23.760228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 161/318] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:23.817006 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-c5b64e33-87e2-42b0-80c4-55b2f5a3bd80 req-a90ed6ef-617c-497f-aedd-0080dcb522ef service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "63e5af09-dbe0-471f-a5bd-9519f11c8d70", "tag": "2d8c367e-4bc1-45e8-8730-7a0f3f231689"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:23.828899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c5b64e33-87e2-42b0-80c4-55b2f5a3bd80 req-a90ed6ef-617c-497f-aedd-0080dcb522ef service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:23.829421 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c5b64e33-87e2-42b0-80c4-55b2f5a3bd80 req-a90ed6ef-617c-497f-aedd-0080dcb522ef service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:23.830015 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c5b64e33-87e2-42b0-80c4-55b2f5a3bd80 req-a90ed6ef-617c-497f-aedd-0080dcb522ef service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:23.843469 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-c5b64e33-87e2-42b0-80c4-55b2f5a3bd80 req-a90ed6ef-617c-497f-aedd-0080dcb522ef service nova] Creating event network-changed:2d8c367e-4bc1-45e8-8730-7a0f3f231689 for instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 on hv-cinder-maste Sep 25 00:58:23.853451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-c5b64e33-87e2-42b0-80c4-55b2f5a3bd80 req-a90ed6ef-617c-497f-aedd-0080dcb522ef service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042275 Sep 25 00:58:23.854049 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 162/319] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:23 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:58:23.882803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:23.885695 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:23.894583 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-25e4db03-afae-4101-9e70-fa4a8841822d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.199784 Sep 25 00:58:23.900303 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 158/320] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:23 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:24.136083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:24.137937 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:24.150068 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:24.150748 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:24.151298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:24.153202 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:24.153840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:24.154474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:24.423476 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:24.425870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:24.432823 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:24.433504 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:24.434114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:24.435308 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:24.435919 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:24.436338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:24.606064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:24.617205 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:24.618800 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:24.620287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-38005133-8a91-414b-b808-539978882d38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.200723 Sep 25 00:58:24.621365 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 159/321] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:24 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:24.651016 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:24.651637 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:24.658127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:24.659424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:24.660047 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:24.660956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:24.661697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:24.662289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:24.817632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:24.818138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:24.825980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-41ec1d31-4a96-4306-a800-9c099c9feea6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.178698 Sep 25 00:58:24.826689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 160/322] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:24 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:24.844257 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9713887d-82b2-444b-bb26-64bbf2547190 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.716719 Sep 25 00:58:24.845115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 163/323] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:24 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 720 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:24.926977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:24.927864 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:24.934128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:24.935037 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:24.935655 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:24.936537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:24.937119 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:24.937579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:25.093818 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:25.103802 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-83fff862-1b73-49b2-8a2f-9723b82dc8ef tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.183491 Sep 25 00:58:25.104660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 161/324] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:24 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:25.649445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:25.650373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:25.657518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:25.658079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:25.658580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:25.659707 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:25.660228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:25.660717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:25.667627 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:25.816271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:25.824766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:25.826238 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:25.827398 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-17167147-ad56-4606-b3b6-0407fa349ebc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.184344 Sep 25 00:58:25.827946 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 164/325] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:25 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:25.853406 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:25.854817 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:25.861048 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:25.861643 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:25.862700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:25.864454 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:25.864926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:25.865527 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:25.870344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:25.871046 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:25.877717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:25.878451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:25.878723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:25.882134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:25.882615 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:25.883201 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:26.052854 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:26.061663 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:26.065423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-78aedc3d-0ed5-4e2f-a059-3f880279d7a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.199293 Sep 25 00:58:26.071376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7ebdafa9-d75a-44ad-8f93-6f45739b96ea tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.223241 Sep 25 00:58:26.072000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 162/326] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:25 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:26.093281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 165/327] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:25 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 217 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:26.145824 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:26.146775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:26.153174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:26.153756 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:26.154189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:26.154946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:26.155413 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:26.155900 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:26.308532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:26.318222 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2e57dfed-fb47-4b37-a0f6-40acdbff910b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.193717 Sep 25 00:58:26.319351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 163/328] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:26 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:26.854364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:26.854980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:26.857540 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:26.864917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:26.865513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:26.865953 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:26.866643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:26.867097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:26.867517 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:27.014205 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:27.023526 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:27.025130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:27.026492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-acc5cbc1-7176-4641-b393-971225235a70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.177074 Sep 25 00:58:27.027174 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 166/329] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:26 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:27.089689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:27.090562 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:27.101424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:27.102030 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:27.102534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:27.103305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:27.103829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:27.103943 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:27.104335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:27.104477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:27.109718 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=16 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:27.113038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:27.113584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:27.114020 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:27.115116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:27.115581 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:27.116058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:27.364068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:27.376176 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5c0687d4-de1b-4d4d-ac79-cf4d9f18f214 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.290289 Sep 25 00:58:27.381483 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 164/330] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:27 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 292 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:27.390680 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:27.391299 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:27.448237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:27.448712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:27.449168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:27.453146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:27.453822 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:27.454285 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:27.606571 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:27.626369 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-772c6576-eefc-47cd-9003-772c5d1085ac tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.526106 Sep 25 00:58:27.642350 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 167/331] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:27 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 529 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:28.039359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:28.055188 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:28.055969 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:28.059289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1ab76a5a-7c79-4e03-8ec9-0a1c0cb22ab2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.675742 Sep 25 00:58:28.076985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 165/332] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:27 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 682 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:28.079529 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:28.086454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:28.087393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:28.089968 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:28.090628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:28.091288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:28.400761 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:28.401400 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:28.419257 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:28.419756 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:28.420191 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:28.421063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:28.421715 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:28.422298 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:28.428359 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:28.445948 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:28.448492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:28.451344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3b0a0a27-1114-4edc-bd3d-29a4cf8e2090 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1746 microversion: 2.20 time: 0.400419 Sep 25 00:58:28.452150 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 168/333] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:28 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1746 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:28.661450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:28.662074 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:28.670824 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:28.680623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:28.681453 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:28.683256 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-473b5bb6-ffef-494f-8f2b-ae1020b20279 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.286507 Sep 25 00:58:28.683601 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:28.684158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 166/334] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:28 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:28.689851 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:28.690707 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:28.693465 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:28.913822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:28.926352 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ffe38f0c-667d-43aa-bbc9-14b949d4aa3f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.278495 Sep 25 00:58:28.927165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 169/335] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:28 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:29.085130 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:29.085769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:29.096178 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:29.097196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:29.100105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:29.101016 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:29.101633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:29.102358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:29.252220 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:29.260893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-564cadc5-f915-4032-84a0-f1111d88e99e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.180031 Sep 25 00:58:29.261885 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 167/336] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:29 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:29.488189 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:29.488885 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:29.495720 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:29.496630 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:29.497210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:29.498317 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:29.498901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:29.499466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:29.662871 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:29.671624 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:29.673449 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:29.675075 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-fae86878-8137-4436-83f9-abdbc102c79e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1746 microversion: 2.20 time: 0.191799 Sep 25 00:58:29.685738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 170/337] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:29 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1746 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:29.712522 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:29.713122 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:29.719607 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:29.720106 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:29.720547 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:29.721570 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:29.721989 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:29.722596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:29.954649 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:29.956215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:29.967152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:29.967692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:29.968129 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:29.972338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:29.973038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:29.977197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:30.001481 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:30.016647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9a46cd15-8a97-4247-99a4-14f926b67ac6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.303539 Sep 25 00:58:30.018391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 168/338] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:29 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:30.170258 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-854a5803-3d90-4a41-b0ca-a96fa410dad3 req-58f7f0ab-d05b-4533-95ef-79ce894cea2b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6892e391-59db-43ba-8843-ce34ade53390", "name": "network-vif-plugged", "status": "completed", "tag": "36da5046-389c-4afa-9d14-41833f5b0db3"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:30.204683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-854a5803-3d90-4a41-b0ca-a96fa410dad3 req-58f7f0ab-d05b-4533-95ef-79ce894cea2b service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:30.213011 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-854a5803-3d90-4a41-b0ca-a96fa410dad3 req-58f7f0ab-d05b-4533-95ef-79ce894cea2b service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:30.216800 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-854a5803-3d90-4a41-b0ca-a96fa410dad3 req-58f7f0ab-d05b-4533-95ef-79ce894cea2b service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:30.228052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:30.243176 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f153dd77-54c8-48b7-8fa9-8b4293602461 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.291945 Sep 25 00:58:30.258135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 171/339] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:29 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:30.287698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-854a5803-3d90-4a41-b0ca-a96fa410dad3 req-58f7f0ab-d05b-4533-95ef-79ce894cea2b service nova] Creating event network-vif-plugged:36da5046-389c-4afa-9d14-41833f5b0db3 for instance 6892e391-59db-43ba-8843-ce34ade53390 on hv-cinder-maste Sep 25 00:58:30.293755 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:30.294353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:30.301471 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-854a5803-3d90-4a41-b0ca-a96fa410dad3 req-58f7f0ab-d05b-4533-95ef-79ce894cea2b service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.137059 Sep 25 00:58:30.303230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 169/340] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:30 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 140 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:30.320962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:30.321514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:30.326738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:30.327508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:30.327985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:30.328426 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:30.573889 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:30.590027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c9a81a23-7b65-4c6b-8df1-4a20fc6bc707 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.308203 Sep 25 00:58:30.591097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 172/341] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:30 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 310 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:30.703017 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:30.703788 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:30.711633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:30.712129 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:30.712891 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:30.714521 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:30.715193 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:30.715629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:30.726484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-157cf566-f5fb-47af-a24b-b11048bce87f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Action: 'create', calling method: .version_select at 0x7fe22db6a430>, body: {"keypair": {"name": "tempest-TestMinimumBasicScenario-600490355", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBorw/RMDt9Sr8g3AnDzfXYMQrAHdbxGpDCSAqkdiRjKGZ9D+3vpBkR4+Bc9abtYUj1yis+1H9kBNvTr5CaBuKUQDLx2lAvWD9vi1RsAcTwndKBiD8Juy2Y8v7D990323w=="}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:30.742739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-157cf566-f5fb-47af-a24b-b11048bce87f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Getting quotas for project b161c2e2dd1f42159dc0d71e8216516c. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:58:30.747481 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-157cf566-f5fb-47af-a24b-b11048bce87f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Getting quotas for user 80dd52f1455f44b587716a41e632a155 and project b161c2e2dd1f42159dc0d71e8216516c. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:58:30.774811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-157cf566-f5fb-47af-a24b-b11048bce87f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 402 microversion: 2.1 time: 0.052905 Sep 25 00:58:30.775728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 173/342] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 00:58:30 2022] POST /compute/v2.1/os-keypairs => generated 402 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:30.875314 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:30.888235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:30.889721 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:30.890281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestMinimumBasicScenario-server-2105039361", "imageRef": "473ca943-1420-487b-b2cd-22b2cf4cbb08", "flavorRef": "452", "key_name": "tempest-TestMinimumBasicScenario-600490355", "networks": [{"uuid": "f9e440d6-f48e-45bb-9a01-3ccd9ccf9488"}]}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:30.891001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c44ffe08-b60d-43a0-b87d-5b8cadfae287 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1746 microversion: 2.20 time: 0.192272 Sep 25 00:58:30.891797 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 170/343] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:30 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1746 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:30.945625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] validate_networks() for [('f9e440d6-f48e-45bb-9a01-3ccd9ccf9488', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:58:31.040297 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:31.040909 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:31.046982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.047497 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.047932 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.048654 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.049738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.050365 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.193494 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:31.201626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-75b3f2c9-793f-4113-9879-155275255dc8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1366 microversion: 2.1 time: 0.165132 Sep 25 00:58:31.202213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 171/344] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:31 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1366 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:31.277606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:31.278218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:31.284915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.285726 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.286166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.287080 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.287745 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.288169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.444216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:58:31.444753 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:58:31.445351 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:58:31.445832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:58:31.446377 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:58:31.455958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.456528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.457041 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.457914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.458366 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.458791 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.476431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Getting quotas for project b161c2e2dd1f42159dc0d71e8216516c. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:58:31.480672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Getting quotas for user 80dd52f1455f44b587716a41e632a155 and project b161c2e2dd1f42159dc0d71e8216516c. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:58:31.490588 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:31.494344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:58:31.496869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='473ca943-1420-487b-b2cd-22b2cf4cbb08',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:58:31.505285 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3ff1c85e-d3e4-4144-91cb-782a861cd4a6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.231488 Sep 25 00:58:31.506120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 172/345] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:31 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 233 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:31.566706 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c708f8cd-ef01-4719-a9fe-b9048c472d64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.680290 Sep 25 00:58:31.567446 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 174/346] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:58:30 2022] POST /compute/v2.1/servers => generated 380 bytes in 682 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 00:58:31.578774 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3c6ad9e8-4e78-42e7-ab77-fcd4f4125538 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:31.579402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3c6ad9e8-4e78-42e7-ab77-fcd4f4125538 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:31.618843 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:31.619476 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:31.626538 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.627594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.628661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.630499 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.631144 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.631777 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.672353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3c6ad9e8-4e78-42e7-ab77-fcd4f4125538 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:31.681220 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3c6ad9e8-4e78-42e7-ab77-fcd4f4125538 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1258 microversion: 2.1 time: 0.106214 Sep 25 00:58:31.682234 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 173/347] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:31 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1258 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:31.849535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:31.863692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8be51e52-fef3-4729-94a1-32db6ca0bb1e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.247490 Sep 25 00:58:31.882305 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 175/348] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:31 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:31.919545 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:31.922375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:31.951412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.956057 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.957648 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:31.958429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:31.958890 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:31.960290 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.131420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:32.142284 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:32.144453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:32.146296 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3cc0268c-633e-417a-9dad-57688fd1a48e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.232347 Sep 25 00:58:32.147126 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 174/349] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:31 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:58:32.225613 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:32.226234 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:32.232324 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.232805 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.233250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.233981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.234435 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.234860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.322196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-274a22a9-2d49-4d3e-8722-20be0b640aa0 req-89ada548-8c60-43fb-90ed-862c41a721ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9", "name": "network-vif-plugged", "status": "completed", "tag": "96168749-3e0f-462f-9aed-adacd4fb28c0"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:32.332275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-274a22a9-2d49-4d3e-8722-20be0b640aa0 req-89ada548-8c60-43fb-90ed-862c41a721ee service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.332757 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-274a22a9-2d49-4d3e-8722-20be0b640aa0 req-89ada548-8c60-43fb-90ed-862c41a721ee service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.333189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-274a22a9-2d49-4d3e-8722-20be0b640aa0 req-89ada548-8c60-43fb-90ed-862c41a721ee service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.344761 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-274a22a9-2d49-4d3e-8722-20be0b640aa0 req-89ada548-8c60-43fb-90ed-862c41a721ee service nova] Creating event network-vif-plugged:96168749-3e0f-462f-9aed-adacd4fb28c0 for instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 on hv-cinder-maste Sep 25 00:58:32.350087 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-274a22a9-2d49-4d3e-8722-20be0b640aa0 req-89ada548-8c60-43fb-90ed-862c41a721ee service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.031486 Sep 25 00:58:32.350691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 175/350] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:32 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 33 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:32.412065 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:32.420026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-921b11ec-5a6d-4ff2-be32-8c817fff78ba tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1556 microversion: 2.1 time: 0.199395 Sep 25 00:58:32.420930 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 176/351] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:32 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1556 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:32.444739 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:32.445438 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:32.451714 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.452463 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.452975 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.453769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.454300 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.454812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.531595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:32.532209 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:32.538998 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3e98a9dd-3f5a-4118-8041-5d1122e0c2b6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.099223 Sep 25 00:58:32.539819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 176/352] 10.222.0.52 () {58 vars in 1275 bytes} [Sun Sep 25 00:58:32 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390/os-interface => generated 285 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:58:32.539944 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.540368 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.547782 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.549717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.550379 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.550969 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.707211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:32.707776 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:32.718457 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.722005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.722438 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.723188 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.723654 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.724078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.800281 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:32.809881 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ee08b66a-058c-4295-9101-d44f08889914 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1254 microversion: 2.1 time: 0.108547 Sep 25 00:58:32.810903 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 177/353] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:32 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1254 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:32.838710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:32.851818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3bf5f429-0e5b-4a74-be7f-ef0d6038e7b7 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.324140 Sep 25 00:58:32.852423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 177/354] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:32 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:32.889183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:32.890184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:32.900511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.901212 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.902218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.902652 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:32.903116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:32.903714 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:32.908688 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:33.091642 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:33.102824 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a579208a-b060-4b4c-9aef-11a803a826ce tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.217555 Sep 25 00:58:33.114131 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 178/355] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:32 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:33.176310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:33.177092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:33.183934 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:33.185112 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:33.186382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:33.187300 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:33.187918 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:33.188763 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:33.560819 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:33.573828 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:33.576397 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:33.578019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ac51f5e0-2bd5-486b-a9a1-1a6e0de4e990 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.406428 Sep 25 00:58:33.593781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 178/356] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:33 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 423 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:33.839789 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:33.840771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:33.851490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:33.852262 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:33.852913 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:33.853880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:33.854571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:33.855225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:33.874602 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:33.875201 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:33.891158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:33.902483 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:33.905454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:33.912602 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:33.913912 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:33.915627 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:34.035474 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:34.044453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7902ca6f-a5a3-44a2-a2ce-743c91dd8c64 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1318 microversion: 2.1 time: 0.208912 Sep 25 00:58:34.047319 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 179/357] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:33 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1318 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:34.130413 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:34.131207 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:34.151751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:34.153177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:34.158823 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:34.161504 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:34.162410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:34.163036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:34.399453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:34.413593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1d2f3fcc-81df-4ea0-ad2d-16375d622f94 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.288142 Sep 25 00:58:34.415400 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 180/358] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:34 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 291 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:34.607100 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:34.607890 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:34.620632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:34.621146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:34.623226 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:34.626302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:34.628116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:34.628633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:34.660841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:34.681867 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9171b615-1e3e-424a-9092-f1c277898f78 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.810936 Sep 25 00:58:34.682455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 179/359] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:33 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 812 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:34.754479 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-76dbaa70-8460-4a78-8dd4-e93ea658e264 req-40193919-59a1-47b4-9687-bae7f837f56b service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "6892e391-59db-43ba-8843-ce34ade53390", "tag": "36da5046-389c-4afa-9d14-41833f5b0db3"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:34.769425 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-76dbaa70-8460-4a78-8dd4-e93ea658e264 req-40193919-59a1-47b4-9687-bae7f837f56b service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:34.770328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-76dbaa70-8460-4a78-8dd4-e93ea658e264 req-40193919-59a1-47b4-9687-bae7f837f56b service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:34.770974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-76dbaa70-8460-4a78-8dd4-e93ea658e264 req-40193919-59a1-47b4-9687-bae7f837f56b service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:34.806974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-76dbaa70-8460-4a78-8dd4-e93ea658e264 req-40193919-59a1-47b4-9687-bae7f837f56b service nova] Creating event network-changed:36da5046-389c-4afa-9d14-41833f5b0db3 for instance 6892e391-59db-43ba-8843-ce34ade53390 on hv-cinder-maste Sep 25 00:58:34.819785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-76dbaa70-8460-4a78-8dd4-e93ea658e264 req-40193919-59a1-47b4-9687-bae7f837f56b service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.071337 Sep 25 00:58:34.828079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 180/360] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:34 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:34.882893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:34.898263 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:34.899926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:34.901377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a48ac257-cd44-40c8-b979-f63d48231a63 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.301371 Sep 25 00:58:34.902460 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 181/361] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:34 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:58:35.071951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:35.073507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:35.096441 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:35.097414 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:35.105725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:35.112261 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:35.114386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:35.115960 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:35.276545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:35.292168 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b9c40a3d-7464-4419-9d06-879b748b353f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1316 microversion: 2.1 time: 0.224279 Sep 25 00:58:35.295683 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 181/362] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:35 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1316 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:35.437460 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:35.438217 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:35.457664 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:35.458279 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:35.458865 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:35.459820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:35.460328 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:35.461076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:35.637375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:35.650349 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b6a1d78a-4cba-4397-8112-3fdf7bd39601 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.216585 Sep 25 00:58:35.668032 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 182/363] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:35 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:35.709498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:35.710586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:35.725580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:35.727452 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:35.734016 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:35.734893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:35.735377 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:35.735812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:35.936069 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:35.936964 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:35.951200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:35.954260 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:35.956299 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:35.959199 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:35.960590 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:35.961884 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:36.384645 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:36.395625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c71732af-dec2-4e2c-84b6-17b59415c000 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.691663 Sep 25 00:58:36.400963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 182/364] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:35 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 698 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:36.415181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:36.415796 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:36.427556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:36.446450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:36.447233 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:36.448578 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:36.451085 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:36.453483 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:36.454283 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:36.456234 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d1e01c12-dde4-4e47-b5fa-4ec3b6950b20 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.529176 Sep 25 00:58:36.458204 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:36.458766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:36.463545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:36.470820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 183/365] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:35 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:36.677403 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:36.678675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:36.715301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:36.716303 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:36.716744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:36.736766 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:36.738799 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.020s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:36.739787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:36.815376 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:36.828338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-53dcfea0-9327-417f-a96a-49f62f2666c8 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1316 microversion: 2.1 time: 0.424660 Sep 25 00:58:36.828986 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 183/366] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:36 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1316 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:37.124871 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:37.135808 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-b6e872d4-2b1c-490d-8345-808b8890cfc0 req-e45222a7-ebed-4e0a-aeff-ca4daa930ff5 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "621dfc45-048c-452f-86ca-ffdaeeb5f919", "name": "network-vif-plugged", "status": "completed", "tag": "a554103e-40f9-42b8-b04a-4fc7d8888240"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:37.137718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-edb3106d-e3d7-4e76-a827-c75d883d86c1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.466333 Sep 25 00:58:37.138812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 184/367] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:36 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 468 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:37.160450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b6e872d4-2b1c-490d-8345-808b8890cfc0 req-e45222a7-ebed-4e0a-aeff-ca4daa930ff5 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:37.161196 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b6e872d4-2b1c-490d-8345-808b8890cfc0 req-e45222a7-ebed-4e0a-aeff-ca4daa930ff5 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:37.161938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b6e872d4-2b1c-490d-8345-808b8890cfc0 req-e45222a7-ebed-4e0a-aeff-ca4daa930ff5 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:37.209049 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-b6e872d4-2b1c-490d-8345-808b8890cfc0 req-e45222a7-ebed-4e0a-aeff-ca4daa930ff5 service nova] Creating event network-vif-plugged:a554103e-40f9-42b8-b04a-4fc7d8888240 for instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 on hv-cinder-maste Sep 25 00:58:37.226115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-b6e872d4-2b1c-490d-8345-808b8890cfc0 req-e45222a7-ebed-4e0a-aeff-ca4daa930ff5 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.093748 Sep 25 00:58:37.232154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 184/368] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:37.459496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:37.460795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:37.480506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:37.482404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:37.483073 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:37.484166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:37.484907 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:37.486618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:37.500096 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:37.501189 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:37.519059 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:37.530138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:37.532218 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.011s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:37.535424 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:37.539892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:37.543758 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:37.848512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:37.860973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-531f51f6-1c01-4d39-8238-613d11a35875 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.439566 Sep 25 00:58:37.863195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 185/369] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:37 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 443 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:37.874730 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:37.875740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:37.895160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:37.895671 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:37.896184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:37.902272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:37.902744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:37.903436 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:38.224749 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:38.271039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b5c6d57f-d5cb-4dc1-ab06-9495f30d6782 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.406328 Sep 25 00:58:38.274860 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 186/370] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:37 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 411 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:38.300797 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:38.301999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:38.335703 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:38.336388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:38.337763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:38.341424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:38.344337 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:38.346227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:38.556716 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:38.574961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:38.576433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:38.598652 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4e2f3c97-5037-4d04-a98d-6843b9ce1f87 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 1.083281 Sep 25 00:58:38.601925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 185/371] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:37 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 1108 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:38.837326 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:38.858334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1a173982-be9a-40af-9ca1-92942a6dbcad tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.580768 Sep 25 00:58:38.911631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:38.923054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:38.932300 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 187/372] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:38 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 657 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:38.950337 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:38.956347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:38.957179 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:38.958750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:38.962837 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:38.963495 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:39.305123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:39.305999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:39.316136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:39.317065 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:39.317800 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:39.318619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:39.319335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:39.319869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:39.320829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:39.331975 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5b0e1740-63a2-4ff3-8c2c-1d97233f4419 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.440243 Sep 25 00:58:39.351555 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 186/373] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:38 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 460 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:39.368527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-3cd033a4-aebc-4b5e-83b1-0d9d7c34b8cd req-82adab53-59af-4495-b40b-de649d50bb91 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27675569-1b42-49db-a883-3ad6e55fc75d", "tag": "0940e2bc-bbba-4cec-82c5-38b72d995723"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:39.394219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-3cd033a4-aebc-4b5e-83b1-0d9d7c34b8cd req-82adab53-59af-4495-b40b-de649d50bb91 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:39.395373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-3cd033a4-aebc-4b5e-83b1-0d9d7c34b8cd req-82adab53-59af-4495-b40b-de649d50bb91 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:39.399356 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-3cd033a4-aebc-4b5e-83b1-0d9d7c34b8cd req-82adab53-59af-4495-b40b-de649d50bb91 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:39.434307 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-3cd033a4-aebc-4b5e-83b1-0d9d7c34b8cd req-82adab53-59af-4495-b40b-de649d50bb91 service nova] Creating event network-changed:0940e2bc-bbba-4cec-82c5-38b72d995723 for instance 27675569-1b42-49db-a883-3ad6e55fc75d on hv-cinder-maste Sep 25 00:58:39.441750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-3cd033a4-aebc-4b5e-83b1-0d9d7c34b8cd req-82adab53-59af-4495-b40b-de649d50bb91 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.087848 Sep 25 00:58:39.447112 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 187/374] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 94 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:58:39.607691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:39.617846 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ee3f0b7e-ae80-4d97-8754-c5a1ba2e8bb1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.317627 Sep 25 00:58:39.634744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 188/375] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:39 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 331 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:39.638955 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:39.641143 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:39.648961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:39.649757 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:39.650404 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:39.651371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:39.663485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:39.665073 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.013s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:39.941149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:39.941864 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:39.958107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:39.960497 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:39.960961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:39.961726 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:39.962191 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:39.969675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:40.023460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:40.033378 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:40.034828 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:40.047127 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-11d03cb2-e3d5-4b5b-a289-274e16da109b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.416071 Sep 25 00:58:40.051137 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 188/376] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:39 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 418 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:40.286442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:40.294563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1d21e4ea-d194-4c9f-9d53-55f1babc1168 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1371 microversion: 2.1 time: 0.360110 Sep 25 00:58:40.302408 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 189/377] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:39 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1371 bytes in 368 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:40.364434 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:40.365176 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:40.380793 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:40.381573 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:40.382192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:40.383069 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:40.383682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:40.384263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:40.623392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:40.642750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:40.644190 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:40.648041 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5f05561d-0d10-4bde-b263-7b453f19371b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.287719 Sep 25 00:58:40.662283 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 189/378] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:40 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 302 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:40.670530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:40.671225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:40.671828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:40.672717 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:40.673907 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:40.674496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:40.957617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:40.971472 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-adb7f034-ef01-41ed-8e04-303e6ab9348d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.331383 Sep 25 00:58:40.987818 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 190/379] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:40 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:41.075780 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:41.076482 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:41.089744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:41.090634 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:41.091076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:41.091785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:41.092241 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:41.092676 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:41.359523 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:41.364805 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:41.386190 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:41.386818 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:41.387360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:41.388192 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:41.388846 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:41.389465 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:41.472021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:41.488541 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=10 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:41.519219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:41.530385 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:41.532282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:41.534483 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-82dc0e35-7ae2-4805-9e35-014a8b8d195b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.462581 Sep 25 00:58:41.535241 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 190/380] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:41 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 464 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:41.674612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:41.682489 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:41.685867 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d8b8f132-ba01-4da2-b474-ef1b93dc1d18 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1566 microversion: 2.1 time: 0.362620 Sep 25 00:58:41.686744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:41.687679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 191/381] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:41 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1566 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:41.699297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:41.715388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:41.715889 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:41.716106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:41.716783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:41.717067 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:41.718033 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:41.718723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:41.719396 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:41.737345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:41.737845 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:41.738292 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:41.739012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:41.739480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:41.739909 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:41.856623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-020415df-8011-441b-bb66-d9dae542eddf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.145694 Sep 25 00:58:41.858428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 192/382] 10.222.0.52 () {58 vars in 1275 bytes} [Sun Sep 25 00:58:41 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-interface => generated 285 bytes in 148 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:41.925997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:41.941125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-fcfcbaa3-9a7b-4bf0-88c2-fefc47b9518a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.261530 Sep 25 00:58:41.942093 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 191/383] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:41 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 263 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:41.998509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:41.999828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:42.014301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:42.018383 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:42.019202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:42.021328 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:42.025854 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:42.026575 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:42.579339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:42.580197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:42.610507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:42.611614 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:42.612320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:42.617447 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:42.618167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:42.618810 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:42.863451 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:42.875820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0122f494-eab9-45b1-b22f-054acf7d34e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.881504 Sep 25 00:58:42.882950 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 193/384] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:41 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 889 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:42.888525 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-797e5e42-8235-4c77-b5b4-321011308dec req-479383a6-d3a2-4334-b8b0-a8278ef180ae service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "63e5af09-dbe0-471f-a5bd-9519f11c8d70", "name": "network-vif-plugged", "status": "completed", "tag": "2d8c367e-4bc1-45e8-8730-7a0f3f231689"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:42.916134 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-797e5e42-8235-4c77-b5b4-321011308dec req-479383a6-d3a2-4334-b8b0-a8278ef180ae service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:42.920513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-797e5e42-8235-4c77-b5b4-321011308dec req-479383a6-d3a2-4334-b8b0-a8278ef180ae service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:42.921494 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-797e5e42-8235-4c77-b5b4-321011308dec req-479383a6-d3a2-4334-b8b0-a8278ef180ae service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:42.954128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-797e5e42-8235-4c77-b5b4-321011308dec req-479383a6-d3a2-4334-b8b0-a8278ef180ae service nova] Creating event network-vif-plugged:2d8c367e-4bc1-45e8-8730-7a0f3f231689 for instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 on hv-cinder-maste Sep 25 00:58:42.961806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-797e5e42-8235-4c77-b5b4-321011308dec req-479383a6-d3a2-4334-b8b0-a8278ef180ae service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.077090 Sep 25 00:58:42.962845 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 194/385] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:58:42.974316 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:42.977658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:42.991185 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:42.993962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:42.994719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:42.998603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:42.999879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:43.000544 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:43.331327 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:43.340971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:43.342513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:43.343658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0fa2ed14-1626-4423-9fea-17aa1239f85b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.769196 Sep 25 00:58:43.344224 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 192/386] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:42 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 770 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:43.881120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:43.883598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:43.899777 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b57eb2b9-9cb2-4171-991f-52453cf111b3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.935319 Sep 25 00:58:43.914836 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 195/387] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:42 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 951 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:43.918536 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:43.919403 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:43.927390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=96,nova_api:INSERT=4 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:43.929251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:43.935812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:43.936524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:43.937500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:43.938119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:43.942137 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:44.212951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:44.225931 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9fa29739-7b38-40be-bf07-b5b3afb4132f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.310922 Sep 25 00:58:44.256645 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 193/388] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:43 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:44.380157 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:44.381570 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:44.400151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:44.404478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:44.406654 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:44.410249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:44.414402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:44.423715 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:44.651244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-56ea1296-5ba3-48ad-967e-9d00ba8f9081 req-54adda40-53b2-4e2c-95bc-4007db16b2c5 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "621dfc45-048c-452f-86ca-ffdaeeb5f919", "tag": "a554103e-40f9-42b8-b04a-4fc7d8888240"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:44.682887 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-56ea1296-5ba3-48ad-967e-9d00ba8f9081 req-54adda40-53b2-4e2c-95bc-4007db16b2c5 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:44.683650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-56ea1296-5ba3-48ad-967e-9d00ba8f9081 req-54adda40-53b2-4e2c-95bc-4007db16b2c5 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:44.688261 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-56ea1296-5ba3-48ad-967e-9d00ba8f9081 req-54adda40-53b2-4e2c-95bc-4007db16b2c5 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:44.716587 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-56ea1296-5ba3-48ad-967e-9d00ba8f9081 req-54adda40-53b2-4e2c-95bc-4007db16b2c5 service nova] Creating event network-changed:a554103e-40f9-42b8-b04a-4fc7d8888240 for instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 on hv-cinder-maste Sep 25 00:58:44.739937 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-56ea1296-5ba3-48ad-967e-9d00ba8f9081 req-54adda40-53b2-4e2c-95bc-4007db16b2c5 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.090091 Sep 25 00:58:44.743465 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 194/389] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:44 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:44.869653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:44.894092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:44.895844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:44.903008 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-224e4a02-12b3-419d-b06b-e9ce67ccc406 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.522624 Sep 25 00:58:44.906182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 196/390] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:44 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 532 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:44.930026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:44.930846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:44.944977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:44.948043 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:44.948557 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:44.950028 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:44.950720 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:44.951816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:45.224629 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:45.238112 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b878daf3-d655-42b4-a12e-5809682bc30d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.313074 Sep 25 00:58:45.239339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 195/391] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:44 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:45.253552 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:45.254183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:45.275216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:45.275974 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:45.276714 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:45.277814 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:45.278559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:45.279175 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:45.644631 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:45.681711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d1f4b160-392f-467a-a964-4384a4b38267 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.432724 Sep 25 00:58:45.700789 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 197/392] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:45 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 452 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:45.934182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:45.935564 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:45.957894 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:45.958413 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:45.970388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:45.974595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:45.975153 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:45.975614 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:46.276877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:46.286065 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:46.287236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:46.288200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:46.291700 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:46.299460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c39792a8-b6e9-4b1a-a435-af90314e3c35 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.369372 Sep 25 00:58:46.300437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 196/393] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:45 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 371 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:46.321873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:46.335761 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.014s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:46.336934 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:46.340015 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:46.341683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:46.348392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:46.694473 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:46.709720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b8ed54a6-643f-4b3e-a482-51eafcf1605f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1265 microversion: 2.1 time: 0.436196 Sep 25 00:58:46.716125 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 198/394] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:46 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1265 bytes in 444 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:46.719435 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:46.720906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:46.744884 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:46.745791 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:46.751175 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:46.752355 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:46.753056 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:46.753942 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:47.072561 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:47.087864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bd0f481f-8dd7-4891-a3e7-0e8d16dc6f61 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.372545 Sep 25 00:58:47.091264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 197/395] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:46 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:47.335492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:47.335856 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:47.353889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:47.354868 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:47.355404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:47.356720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:47.363771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:47.364613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:47.599061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:47.613338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:47.615748 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:47.618528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c0b5aa0c-118b-49de-96fd-a03923b26a3d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.286975 Sep 25 00:58:47.619949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 199/396] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:47 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 290 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:58:47.753694 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:47.754428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:47.763089 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:47.765625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:47.766334 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:47.769526 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:47.770275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:47.771050 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:48.131811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:48.132448 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:48.155227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:48.155753 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:48.156203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:48.156943 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:48.162016 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:48.162525 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:48.168511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:48.183194 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-276a8efb-3f22-46e1-b2b3-91680bf6c486 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1452 microversion: 2.1 time: 0.433218 Sep 25 00:58:48.202697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 198/397] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:47 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1452 bytes in 453 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:48.229584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:48.230524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:48.239996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:48.240812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:48.241444 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:48.242338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:48.242962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:48.243548 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:48.505720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:48.517812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5ab90895-8556-4c54-98db-d94e57243d18 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.391578 Sep 25 00:58:48.523869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 200/398] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:48 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:48.576535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:48.614996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d4840c32-6051-461a-986b-236cc998eff6 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1452 microversion: 2.1 time: 0.390711 Sep 25 00:58:48.615527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 199/399] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:48 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1452 bytes in 392 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:48.649628 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:48.650239 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:48.672982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:48.674111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:48.676635 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:48.678909 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:48.679743 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:48.682244 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:48.868003 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:48.876782 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:48.879631 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:48.881148 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1fd2c27a-1f5c-4397-a65a-2edfbc18289d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.236036 Sep 25 00:58:48.898331 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 201/400] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:48 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 254 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:49.556711 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:49.557531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:49.564739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:49.565588 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:49.566225 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:49.567128 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:49.567829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:49.568313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:49.875691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:49.887333 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c44ee33a-68f8-461d-ad0b-8e79c1b447eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.334440 Sep 25 00:58:49.913877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:49.915263 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:49.918838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 200/401] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:49 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 366 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:49.922738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:49.924125 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:49.924883 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:49.926016 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:49.926806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:49.927576 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:50.113010 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:50.122727 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:50.124388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:50.125598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3e9b3ec0-fce1-43de-b2fa-a4ced6c18ff2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.217222 Sep 25 00:58:50.126199 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 202/402] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:49 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:50.915295 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:50.917173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:50.926370 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:50.927747 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:50.928475 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:50.929251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:50.929973 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:50.931237 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:50.971105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-dcccaf0f-b52e-4557-85b7-1f9f630f2c4f req-48c6e959-8989-4bd8-b740-28dbe629206e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "63e5af09-dbe0-471f-a5bd-9519f11c8d70", "tag": "2d8c367e-4bc1-45e8-8730-7a0f3f231689"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:50.998573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dcccaf0f-b52e-4557-85b7-1f9f630f2c4f req-48c6e959-8989-4bd8-b740-28dbe629206e service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:50.999544 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dcccaf0f-b52e-4557-85b7-1f9f630f2c4f req-48c6e959-8989-4bd8-b740-28dbe629206e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:51.000988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dcccaf0f-b52e-4557-85b7-1f9f630f2c4f req-48c6e959-8989-4bd8-b740-28dbe629206e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:51.025909 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-dcccaf0f-b52e-4557-85b7-1f9f630f2c4f req-48c6e959-8989-4bd8-b740-28dbe629206e service nova] Creating event network-changed:2d8c367e-4bc1-45e8-8730-7a0f3f231689 for instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 on hv-cinder-maste Sep 25 00:58:51.043577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-dcccaf0f-b52e-4557-85b7-1f9f630f2c4f req-48c6e959-8989-4bd8-b740-28dbe629206e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.078564 Sep 25 00:58:51.044173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 203/403] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:51.155887 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:51.156590 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:51.170490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:51.175095 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:51.175576 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:51.176463 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:51.176933 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:51.177388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:51.179035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:51.187468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:51.193511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e6d0b056-2529-43cb-940b-4147dfa98c65 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.283200 Sep 25 00:58:51.194464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 201/404] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:50 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:51.346471 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:51.354839 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:51.356323 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:51.357529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-10385c29-8de7-46cf-8fed-0f2f38d37d44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.207068 Sep 25 00:58:51.358251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 204/405] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:51 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:58:52.225478 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:52.226681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:52.233869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:52.234418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:52.234856 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:52.236151 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:52.236610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:52.237305 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:52.383943 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:52.384604 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:52.388159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:52.393439 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:52.394269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:52.394909 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:52.395827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:52.396474 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:52.397090 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:52.542316 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:52.551467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d57a1138-58c7-4b55-997f-37a557cb8d92 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.330390 Sep 25 00:58:52.559928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 202/406] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:52 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 339 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:52.598578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:52.609374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:52.610940 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:52.612533 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6589dcc3-0998-4e3a-89e2-8bc8456f81ff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.232016 Sep 25 00:58:52.613121 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 205/407] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:52 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 234 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:58:53.573787 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:53.574400 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:53.581527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:53.582105 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:53.582878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:53.583754 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:53.584373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:53.584826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:53.637899 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:53.638621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:53.645372 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:53.645952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:53.646454 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:53.647228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:53.647755 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:53.648269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:53.753829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:53.761788 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-17dbb6c5-7d2a-4930-b05b-5b70763898f6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.191794 Sep 25 00:58:53.762445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 203/408] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:53 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:53.789156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:53.797444 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:53.799995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:53.801168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0a82c5a9-849f-45c7-9a42-73e47352a984 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1741 microversion: 2.20 time: 0.167257 Sep 25 00:58:53.801758 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 206/409] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:53 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1741 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:58:54.788094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:54.789535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:54.796717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:54.798437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:54.799484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:54.800421 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:54.801053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:54.801941 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:54.826686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:54.827496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:54.837656 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:54.838157 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:54.838657 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:54.839691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:54.840297 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:54.840888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:55.137366 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:55.148560 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:55.150353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:55.152003 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fcdb17ad-f9e4-49c6-8bc7-0891bd48bdc7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1723 microversion: 2.20 time: 0.329267 Sep 25 00:58:55.164940 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 207/410] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:54 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1723 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:58:55.308281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:55.318219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3336abed-c1cc-4c0d-af18-46e71f94f2d3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.536614 Sep 25 00:58:55.319064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 204/411] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:54 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 538 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:56.181399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:56.182155 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:56.188182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:56.188931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:56.189501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:56.190304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:56.190651 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:56.191887 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:56.322443 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-493c7d78-6352-4b14-8a36-741ba53bc648 req-d359d7a6-ca2b-4732-8e39-7eb6505fd7ee service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "27675569-1b42-49db-a883-3ad6e55fc75d", "name": "network-vif-plugged", "status": "completed", "tag": "0940e2bc-bbba-4cec-82c5-38b72d995723"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:56.338765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-493c7d78-6352-4b14-8a36-741ba53bc648 req-d359d7a6-ca2b-4732-8e39-7eb6505fd7ee service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:56.339674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-493c7d78-6352-4b14-8a36-741ba53bc648 req-d359d7a6-ca2b-4732-8e39-7eb6505fd7ee service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:56.341553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-493c7d78-6352-4b14-8a36-741ba53bc648 req-d359d7a6-ca2b-4732-8e39-7eb6505fd7ee service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:56.350865 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:56.359468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-493c7d78-6352-4b14-8a36-741ba53bc648 req-d359d7a6-ca2b-4732-8e39-7eb6505fd7ee service nova] Creating event network-vif-plugged:0940e2bc-bbba-4cec-82c5-38b72d995723 for instance 27675569-1b42-49db-a883-3ad6e55fc75d on hv-cinder-maste Sep 25 00:58:56.363624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:56.366111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:56.367536 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-493c7d78-6352-4b14-8a36-741ba53bc648 req-d359d7a6-ca2b-4732-8e39-7eb6505fd7ee service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048834 Sep 25 00:58:56.367658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-86d0b0c7-d3f6-439c-b134-e195a9402588 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1723 microversion: 2.20 time: 0.190991 Sep 25 00:58:56.368739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 205/413] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:58:56.368896 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 208/412] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:56 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1723 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:56.374841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:56.375779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:56.390094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:56.390577 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:56.391202 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:56.392625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:56.393077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:56.393692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:56.558487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:56.568898 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ee52a26d-3515-480c-bd4d-8ff81203bba6 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.198661 Sep 25 00:58:56.569505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 206/414] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:56 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:57.393956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:57.396896 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:57.413107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:57.413681 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:57.414651 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:57.416097 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:57.416565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:57.417002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:57.594772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:57.595466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:57.604535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:57.605031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:57.605671 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:57.606904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:57.607712 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:57.608417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:57.642876 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:57.651611 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:57.653332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:57.654701 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f68ebcdf-1673-4780-9555-015baed2d010 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1723 microversion: 2.20 time: 0.264786 Sep 25 00:58:57.655246 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 209/415] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:57 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1723 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:57.800675 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:57.808650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f0c86fd5-9a74-4a45-8c3a-776efa022272 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.218077 Sep 25 00:58:57.809228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 207/416] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:57 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:58:58.420534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-85f275c5-646a-4cf3-b77d-2bdbe0dab722 req-5e305eb3-4357-474b-bc9d-e9f1a3c90330 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9", "name": "network-vif-unplugged", "status": "completed", "tag": "96168749-3e0f-462f-9aed-adacd4fb28c0"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:58.432870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-85f275c5-646a-4cf3-b77d-2bdbe0dab722 req-5e305eb3-4357-474b-bc9d-e9f1a3c90330 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:58.433381 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-85f275c5-646a-4cf3-b77d-2bdbe0dab722 req-5e305eb3-4357-474b-bc9d-e9f1a3c90330 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:58.433812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-85f275c5-646a-4cf3-b77d-2bdbe0dab722 req-5e305eb3-4357-474b-bc9d-e9f1a3c90330 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:58.461447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-85f275c5-646a-4cf3-b77d-2bdbe0dab722 req-5e305eb3-4357-474b-bc9d-e9f1a3c90330 service nova] Creating event network-vif-unplugged:96168749-3e0f-462f-9aed-adacd4fb28c0 for instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 on hv-cinder-maste Sep 25 00:58:58.466718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-85f275c5-646a-4cf3-b77d-2bdbe0dab722 req-5e305eb3-4357-474b-bc9d-e9f1a3c90330 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.049802 Sep 25 00:58:58.467328 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 210/417] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:58:58 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:58:58.682291 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:58.683157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:58.689947 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:58.690442 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:58.690959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:58.691722 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:58.692184 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:58.692605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:58.833810 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:58.835226 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:58.836034 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:58.842953 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:58.843058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:58.843529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:58.844039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:58.844759 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:58.844993 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:58.845650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ea009bcb-c8a4-47fd-ab74-63a2deb230bd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1726 microversion: 2.20 time: 0.169309 Sep 25 00:58:58.845921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:58.846414 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:58.846541 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 208/418] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:58 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1726 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:58:58.873558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1cb3dd71-9b2f-4d99-bda4-350239151043"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:58.876911 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:58.892416 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:58.893195 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:58.893677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:58.894633 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:58.907010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:58.908299 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:59.019058 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:59.027200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d2c98166-e8ee-4e91-a24d-b1cb56ff7e01 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1358 microversion: 2.1 time: 0.196426 Sep 25 00:58:59.027809 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 211/419] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:58:58 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1358 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:58:59.244155 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a797b4bd-e8d4-4c85-928e-cccdf72fed0b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.374889 Sep 25 00:58:59.244770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 209/420] 10.222.0.52 () {62 vars in 1364 bytes} [Sun Sep 25 00:58:58 2022] POST /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments => generated 188 bytes in 376 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 00:58:59.404241 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:58:59.404841 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:59.411642 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:59.412135 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:59.412571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:59.413294 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:59.413752 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:59.414174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:59.461090 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120451) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 25 00:58:59.529781 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=99,nova_cell1:UPDATE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:58:59.556965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a9af8f75-8be2-4440-9b99-c505245d2415 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/action" status: 202 len: 0 microversion: 2.20 time: 0.156545 Sep 25 00:58:59.557603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 212/421] 10.222.0.52 () {62 vars in 1319 bytes} [Sun Sep 25 00:58:59 2022] POST /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/action => generated 0 bytes in 158 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 25 00:58:59.567797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:58:59.569250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:58:59.576322 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:59.577049 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:59.577592 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:59.578372 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:58:59.579597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:58:59.580342 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:58:59.757642 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:59.768648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:59.770160 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:58:59.771347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-21d69400-6c9a-4c3a-81e7-0b1e897e0c1e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1812 microversion: 2.20 time: 0.209156 Sep 25 00:58:59.772194 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 210/422] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:58:59 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1812 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:00.051557 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:00.052432 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:00.060289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.061137 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.061705 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.062705 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.063527 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.065538 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.282376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:00.291643 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0f6b6cd5-a846-4c44-a1f5-4abb82d382a1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1548 microversion: 2.1 time: 0.245012 Sep 25 00:59:00.301326 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 213/423] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:00 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1548 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:00.316356 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:00.317007 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:00.325955 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.326566 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.327338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.328092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.328543 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.329019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.516340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:00.525719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3d5690c0-9d40-49e3-90b0-89ab1c4f01d2 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1548 microversion: 2.1 time: 0.214101 Sep 25 00:59:00.526306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 211/424] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:00 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1548 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:59:00.550045 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:00.555082 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:servers:allow_all_filters failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:00.555850 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Searching by: {'deleted': False, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c'} {{(pid=120451) get_all /opt/stack/nova/nova/compute/api.py:3017}} Sep 25 00:59:00.563795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.564411 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.564846 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.565599 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.566047 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.566733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.667159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.multi_cell_list [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Listed batch of 1 results from cell out of 1000 limit. Returned 1 total so far. {{(pid=120451) do_query /opt/stack/nova/nova/compute/multi_cell_list.py:373}} Sep 25 00:59:00.774238 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-41a2028f-da5c-472f-8647-c1fc4c47a97c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers" status: 200 len: 343 microversion: 2.1 time: 0.228656 Sep 25 00:59:00.775016 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 214/425] 10.222.0.52 () {58 vars in 1125 bytes} [Sun Sep 25 00:59:00 2022] GET /compute/v2.1/servers => generated 343 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:59:00.789835 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:00.790450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:00.799053 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:00.799632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.800036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:00.800496 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.802584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.803419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.803900 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.804423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.807117 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.807613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.808098 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.808997 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:00.809506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:00.810145 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:00.955329 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:00.973862 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:00.975228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:00.976926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:00.978169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7b8341ab-dbdd-4064-97f0-cdbec9d0fc6e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1752 microversion: 2.20 time: 0.184349 Sep 25 00:59:00.978739 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 215/426] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:00 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1752 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:00.991841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e9217294-e018-45f1-829f-ccfef12bd38f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1548 microversion: 2.1 time: 0.206031 Sep 25 00:59:00.992420 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 212/427] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:00 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1548 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:02.003197 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:02.003808 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:02.010046 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:02.010529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:02.010961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:02.011704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:02.012149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:02.012568 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:02.153147 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:02.162652 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:02.164102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:02.165379 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d623c1d5-9799-4583-b623-a85da8a78921 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1754 microversion: 2.20 time: 0.166389 Sep 25 00:59:02.165951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 216/428] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:01 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1754 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:02.677121 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "a2c4fd58-bcf5-4e8b-ac9d-746e34799641"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:02.679677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:02.686594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:02.687071 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:02.687507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:02.688830 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:02.689552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:02.689982 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:02.995485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-84b11e36-98d9-4209-942c-99a14c9ba4e5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "POST /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.323316 Sep 25 00:59:02.996094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 213/429] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 00:59:02 2022] POST /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments => generated 194 bytes in 325 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:03.191863 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:03.192576 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:03.199156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:03.199659 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:03.200095 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:03.200785 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:03.201237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:03.201694 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:03.338997 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:03.348160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:03.349643 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:03.350795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4f181d2e-337b-430a-9ba8-c916a77c3e2c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.162965 Sep 25 00:59:03.351344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 217/430] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:03 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:04.378981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:04.379689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:04.386325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:04.386817 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:04.387265 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:04.387987 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:04.388440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:04.388869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:04.539224 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:04.547985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:04.549551 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:04.551080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-be6c00df-84a0-4676-b12f-5e05987486ba tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.177590 Sep 25 00:59:04.551603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 214/431] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:04 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:05.576223 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:05.577056 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:05.584367 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:05.584950 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:05.585492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:05.586322 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:05.586870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:05.587448 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:05.753811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:05.764715 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:05.766366 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:05.768774 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-92a83c95-ac1b-45df-b857-d6c4c60e1b90 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.196470 Sep 25 00:59:05.769375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 218/432] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:05 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:06.799520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:06.800133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:06.809390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:06.809894 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:06.810329 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:06.811223 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:06.811491 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:06.811914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:06.846209 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=90,nova_cell1:INSERT=4,nova_cell1:UPDATE=6 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:06.993798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:07.004779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:07.006374 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:07.007888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7c1f9cad-9200-452a-a322-0063e67d95df tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.213487 Sep 25 00:59:07.010164 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 215/433] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:06 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:08.037460 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:08.038073 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:08.045198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:08.045791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:08.046452 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:08.048549 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:08.053223 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:08.054105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:08.293934 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:08.303539 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:08.305601 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:08.306910 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e735920e-78d8-49ec-a10b-e9cf9d848d1c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.273456 Sep 25 00:59:08.307610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 219/434] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:08 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 275 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:09.343574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:09.345446 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:09.353184 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:09.353965 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:09.354569 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:09.355437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:09.356040 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:09.356623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:09.566470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:09.576187 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:09.577675 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:09.579264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-998bf83e-e620-4e5f-8595-cc99e86ce4d8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.239546 Sep 25 00:59:09.579870 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 216/435] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:09 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:10.608559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:10.609402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:10.612034 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:10.618698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:10.619454 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:10.619888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:10.620590 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:10.621069 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:10.621562 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:10.762983 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:10.774173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:10.775699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:10.776909 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-08916074-59dd-408e-b5cb-1273db842aa8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.172307 Sep 25 00:59:10.777498 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 220/436] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:10 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:11.804985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:11.805628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:11.813249 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:11.814106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:11.814677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:11.816335 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:11.817170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:11.818290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:11.970842 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:11.980513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:11.982607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:11.984274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1f20ae0f-89f6-494a-a381-41e285f7f26e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.184066 Sep 25 00:59:11.986433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 217/437] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:11 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:13.011211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:13.012746 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:13.020906 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:13.021955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:13.022623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:13.023632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:13.024467 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:13.025612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:13.224477 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:13.240133 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:13.241893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:13.243293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f6577f2d-e739-47e0-979d-35005259323a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.236765 Sep 25 00:59:13.244094 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 221/438] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:13 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 238 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:14.276589 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:14.277214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:14.279849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=50 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:14.287080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:14.287569 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:14.288000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:14.288697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:14.289348 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:14.289775 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:14.431690 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:14.441670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:14.443165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:14.444325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5fb00a71-3a2e-4631-9c82-a155b5ec147f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.172320 Sep 25 00:59:14.444874 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 218/439] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:14 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:15.473792 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:15.474418 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:15.480482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:15.480961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:15.481427 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:15.482116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:15.482560 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:15.482986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:15.617382 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:15.628169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:15.629628 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:15.630802 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4c74938e-8e0e-4ca8-b597-dd6dbb7077bf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.161152 Sep 25 00:59:15.631364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 222/440] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:15 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:16.656606 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:16.657623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:16.664178 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:16.664685 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:16.665137 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:16.665864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:16.666329 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:16.666746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:16.803760 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:16.812354 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:16.813823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:16.814966 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-42f2038f-afef-4f80-af53-703711ef4258 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.162818 Sep 25 00:59:16.815518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 219/441] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:16 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:17.124322 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:17.125023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:17.131377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:17.131893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:17.132324 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:17.133056 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:17.133526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:17.133944 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:17.268918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:17.277497 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1db7cb3b-a872-454d-aaf1-64681621e62d tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1594 microversion: 2.1 time: 0.156966 Sep 25 00:59:17.278055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 223/442] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:17 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1594 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:17.840605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:17.841074 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:17.848459 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:17.849026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:17.849559 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:17.850252 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:17.850703 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:17.851124 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:18.083943 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:18.092773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:18.095042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:18.096212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ac4ebffd-6abc-437c-9590-17b453167a3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.259848 Sep 25 00:59:18.096766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 220/443] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:17 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:19.123597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:19.124284 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:19.141151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:19.142002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:19.142437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:19.143140 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:19.143596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:19.144017 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:19.188440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-180a7932-2961-4879-b3ee-bea5c1213efd req-43e4b0a6-e480-4c83-bfa5-8a19832bf32f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27675569-1b42-49db-a883-3ad6e55fc75d", "tag": "0940e2bc-bbba-4cec-82c5-38b72d995723"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:19.207020 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-180a7932-2961-4879-b3ee-bea5c1213efd req-43e4b0a6-e480-4c83-bfa5-8a19832bf32f service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:19.207498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-180a7932-2961-4879-b3ee-bea5c1213efd req-43e4b0a6-e480-4c83-bfa5-8a19832bf32f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:19.207928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-180a7932-2961-4879-b3ee-bea5c1213efd req-43e4b0a6-e480-4c83-bfa5-8a19832bf32f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:19.231173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-180a7932-2961-4879-b3ee-bea5c1213efd req-43e4b0a6-e480-4c83-bfa5-8a19832bf32f service nova] Creating event network-changed:0940e2bc-bbba-4cec-82c5-38b72d995723 for instance 27675569-1b42-49db-a883-3ad6e55fc75d on hv-cinder-maste Sep 25 00:59:19.238068 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-180a7932-2961-4879-b3ee-bea5c1213efd req-43e4b0a6-e480-4c83-bfa5-8a19832bf32f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053858 Sep 25 00:59:19.242951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 221/444] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:59:19 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:59:19.420316 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:19.431057 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:19.433749 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:19.435354 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ef774b3a-8cab-4b15-873b-73e555cfe2a3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.317015 Sep 25 00:59:19.436392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 224/445] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:19 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 318 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:20.469309 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:20.470128 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:20.477193 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:20.477907 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:20.478409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:20.479285 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:20.479873 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:20.480531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:20.681702 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:20.698821 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:20.700520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:20.701929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b16aaaaa-b28e-4942-9d79-a46596e5a4b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.239936 Sep 25 00:59:20.702656 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 222/446] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:20 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:20.744059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Action: 'action', calling method: >, body: {"addSecurityGroup": {"name": "tempest-secgroup-smoke-211599788"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:20.744608 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:20.751992 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:20.752409 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:20.752839 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:20.753563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:20.754015 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:20.754430 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:20.889595 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.network.security_group_api [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Adding security group 42581346-2174-404e-9723-a7e42899addf to port 0940e2bc-bbba-4cec-82c5-38b72d995723 Sep 25 00:59:21.260889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-25402f01-5e14-49fe-9597-c5d80dd34e8f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "POST /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/action" status: 202 len: 0 microversion: 2.1 time: 0.521219 Sep 25 00:59:21.261739 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 225/447] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 00:59:20 2022] POST /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/action => generated 0 bytes in 523 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 00:59:21.271435 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:21.272053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:21.278285 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-87b1cb1a-66dc-4d1d-87b1-c500ec2f6765 req-3b2d541b-0e04-430b-95c6-0cac39d7d324 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27675569-1b42-49db-a883-3ad6e55fc75d", "tag": "0940e2bc-bbba-4cec-82c5-38b72d995723"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:21.290479 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:21.290990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:21.294587 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:21.295389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:21.295909 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:21.301605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:21.305448 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-87b1cb1a-66dc-4d1d-87b1-c500ec2f6765 req-3b2d541b-0e04-430b-95c6-0cac39d7d324 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:21.306664 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-87b1cb1a-66dc-4d1d-87b1-c500ec2f6765 req-3b2d541b-0e04-430b-95c6-0cac39d7d324 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:21.307117 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-87b1cb1a-66dc-4d1d-87b1-c500ec2f6765 req-3b2d541b-0e04-430b-95c6-0cac39d7d324 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:21.333274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-87b1cb1a-66dc-4d1d-87b1-c500ec2f6765 req-3b2d541b-0e04-430b-95c6-0cac39d7d324 service nova] Creating event network-changed:0940e2bc-bbba-4cec-82c5-38b72d995723 for instance 27675569-1b42-49db-a883-3ad6e55fc75d on hv-cinder-maste Sep 25 00:59:21.351905 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-87b1cb1a-66dc-4d1d-87b1-c500ec2f6765 req-3b2d541b-0e04-430b-95c6-0cac39d7d324 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.076404 Sep 25 00:59:21.352803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 226/448] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:59:21 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 79 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:21.504421 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:21.513290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-fa39a33e-1fe3-4eae-b0b1-9530352e527f tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1759 microversion: 2.1 time: 0.245579 Sep 25 00:59:21.514186 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 223/449] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:21 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1759 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:21.735861 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:21.736563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:21.743916 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:21.744410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:21.744967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:21.745614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:21.746075 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:21.746512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:21.886439 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:21.895827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:21.897541 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:21.898715 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0c677537-b991-436c-881a-8ca9a69d9a7e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.168499 Sep 25 00:59:21.899275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 227/450] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:21 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:22.924718 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:22.925356 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:22.931789 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:22.932276 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:22.932713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:22.933572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:22.934033 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:22.934459 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:23.065990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:23.074524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:23.076061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:23.077361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5f4e4bbb-21af-4034-995f-447d1345e680 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.157118 Sep 25 00:59:23.077978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 224/451] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:22 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:24.103058 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:24.103673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:24.105921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=50,nova_api:UPDATE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:24.111237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:24.111977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:24.112415 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:24.113106 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:24.113586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:24.114014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:24.251768 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:24.261148 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:24.262712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:24.264013 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2569589f-ba6e-4086-ac70-246345c4fb58 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.164814 Sep 25 00:59:24.264629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 228/452] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:24 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:25.290766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:25.291365 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:25.298781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:25.299262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:25.299701 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:25.300389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:25.300844 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:25.301285 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:25.445831 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:25.455338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:25.456810 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:25.458053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2e777e03-1275-4e55-979f-048fc68049e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.171175 Sep 25 00:59:25.458603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 225/453] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:25 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:26.455955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-dc7aa4e0-fa53-4a20-86b0-1c1381361911 req-00fc2308-95f1-4b41-bd19-36752f8f1b48 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9", "name": "network-vif-plugged", "status": "completed", "tag": "96168749-3e0f-462f-9aed-adacd4fb28c0"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:26.467693 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dc7aa4e0-fa53-4a20-86b0-1c1381361911 req-00fc2308-95f1-4b41-bd19-36752f8f1b48 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:26.468319 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dc7aa4e0-fa53-4a20-86b0-1c1381361911 req-00fc2308-95f1-4b41-bd19-36752f8f1b48 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:26.468879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-dc7aa4e0-fa53-4a20-86b0-1c1381361911 req-00fc2308-95f1-4b41-bd19-36752f8f1b48 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:26.480797 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-dc7aa4e0-fa53-4a20-86b0-1c1381361911 req-00fc2308-95f1-4b41-bd19-36752f8f1b48 service nova] Creating event network-vif-plugged:96168749-3e0f-462f-9aed-adacd4fb28c0 for instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 on hv-cinder-maste Sep 25 00:59:26.488360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:26.488821 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:26.493775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-dc7aa4e0-fa53-4a20-86b0-1c1381361911 req-00fc2308-95f1-4b41-bd19-36752f8f1b48 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041872 Sep 25 00:59:26.494550 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 229/454] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:59:26 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:26.498074 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:26.499970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:26.500799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:26.504042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:26.504528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:26.504974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:26.659451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:26.674860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:26.676527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:26.678451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0bd6197b-a23d-44c4-a124-d4f63dd9140f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.197592 Sep 25 00:59:26.680742 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 226/455] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:26 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:27.708733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:27.709466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:27.716106 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:27.716593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:27.717032 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:27.717787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:27.718740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:27.719263 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:27.884275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:27.886872 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:INSERT=1,nova_cell1:SELECT=99 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:27.895668 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:27.897266 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:27.898482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-746ef888-902c-44b2-aeaa-bd23cd7223b7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.194091 Sep 25 00:59:27.899049 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 230/456] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:27 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:28.644683 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:28.645240 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:28.651957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.652443 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.652877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.653619 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.654072 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.654495 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.678903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d3fd7d3e-88c1-411e-9387-8048c7c64c32 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390/ips" status: 200 len: 149 microversion: 2.1 time: 0.038649 Sep 25 00:59:28.679649 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 227/457] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 00:59:28 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390/ips => generated 149 bytes in 40 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:28.692127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "28ae3ae1-26a0-462e-8a8f-d89670d28529"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:28.694540 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:28.700922 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.701455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.701894 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.703093 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.703549 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.703984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.793099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Invalid volume: volume 28ae3ae1-26a0-462e-8a8f-d89670d28529 is already attached to instances: af139b53-2db6-43ea-9e69-27dfbb9304bc Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: POST /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390/os-volume_attachments HTTP/1.1 Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 74 Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 00:59:28.794873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {"volumeAttachment": {"volumeId": "28ae3ae1-26a0-462e-8a8f-d89670d28529"}} Sep 25 00:59:28.796506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Invalid volume: volume 28ae3ae1-26a0-462e-8a8f-d89670d28529 is already attached to instances: af139b53-2db6-43ea-9e69-27dfbb9304bc Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'server_id': '6892e391-59db-43ba-8843-ce34ade53390', 'body': {'volumeAttachment': {'volumeId': '28ae3ae1-26a0-462e-8a8f-d89670d28529'}}} Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 28ae3ae1-26a0-462e-8a8f-d89670d28529 is already attached to instances: af139b53-2db6-43ea-9e69-27dfbb9304bc Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:59:28.797428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 28ae3ae1-26a0-462e-8a8f-d89670d28529 is already attached to instances: af139b53-2db6-43ea-9e69-27dfbb9304bc Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:28.798948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:59:28.800450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:59:28.800450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:59:28.800450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:59:28.800450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 28ae3ae1-26a0-462e-8a8f-d89670d28529 is already attached to instances: af139b53-2db6-43ea-9e69-27dfbb9304bc Sep 25 00:59:28.800450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 00:59:28.800450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 400 to user: Invalid volume: volume 28ae3ae1-26a0-462e-8a8f-d89670d28529 is already attached to instances: af139b53-2db6-43ea-9e69-27dfbb9304bc {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:59:28.800450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e5d88c87-18b0-4f08-ab4f-4a53e6a46f95 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.109175 Sep 25 00:59:28.800450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 231/458] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 00:59:28 2022] POST /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390/os-volume_attachments => generated 174 bytes in 110 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 25 00:59:28.806206 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:28.806711 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:28.813203 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.813796 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.814235 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.814932 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.815387 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.815814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.841740 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 00:59:28.926603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:28.928080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-401c4605-dbcd-4b95-94ba-868aac79a6bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 204 len: 0 microversion: 2.1 time: 0.125545 Sep 25 00:59:28.928441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:28.928993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 228/459] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 00:59:28 2022] DELETE /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 00:59:28.938221 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:28.938918 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:28.940676 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.941166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.941669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.942371 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.942824 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.943244 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.944841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.945352 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.945793 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:28.946494 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:28.947223 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:28.947859 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:29.085843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:29.087270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:29.095660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:29.095868 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9c545ac0-7463-46e9-8742-72de78e6be87 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1680 microversion: 2.1 time: 0.162090 Sep 25 00:59:29.096466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 229/460] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:28 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1680 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:29.097290 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:29.098268 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ee60b136-a984-4cb9-a239-721993cd4ab6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.177034 Sep 25 00:59:29.098812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 232/461] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:28 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:30.119682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:30.120517 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:30.124138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:30.125678 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:30.128755 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:30.129270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:30.129709 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:30.130418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:30.130877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:30.131310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:30.131917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:30.132418 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:30.132846 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:30.133641 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:30.134091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:30.134506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:30.266864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:30.269455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:30.274807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e8178a98-eea1-40a0-ad6a-0c757f9e1fc6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1680 microversion: 2.1 time: 0.158916 Sep 25 00:59:30.275387 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 230/462] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:30 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1680 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:30.278287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:30.279765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:30.281651 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-227fcb35-327e-4d1b-b712-af30dd4b7280 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.161569 Sep 25 00:59:30.282309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 233/463] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:30 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:31.299792 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:31.300436 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:31.306973 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:31.307455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:31.307593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:31.308018 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:31.308115 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:31.308572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:31.309112 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:31.309520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:31.313814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:31.315108 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:31.315739 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:31.316502 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:31.317203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:31.317683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:31.318102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:31.460568 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:31.460998 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:31.470242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4879cb68-5198-4576-9646-089403ebd7fb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1680 microversion: 2.1 time: 0.174999 Sep 25 00:59:31.471057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 231/464] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:31 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1680 bytes in 176 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:31.471207 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:31.472410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:31.473670 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3199f52d-86d9-43eb-8ddb-e5d24580a5ab tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.170004 Sep 25 00:59:31.474224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 234/465] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:31 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:32.250217 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "0ff9470c-98a0-4477-980a-1fb9e8d393c5"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:32.252709 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:32.260586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:32.260586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:32.260586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:32.261347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:32.261805 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:32.262349 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:32.494566 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:32.495192 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:32.502215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:32.502701 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:32.503142 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:32.503840 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:32.504289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:32.504709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:32.634299 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:32.642280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-deae4668-0381-495d-89e7-2640732ae493 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1680 microversion: 2.1 time: 0.151659 Sep 25 00:59:32.642833 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 235/466] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:32 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1680 bytes in 153 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:59:32.647264 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:32.647858 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:32.653457 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:32.653937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:32.654368 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:32.655056 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:32.655517 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:32.656416 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:32.788979 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:32.798582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:32.800031 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:32.801191 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b235d0d0-a43a-4550-8b14-2765415a4a65 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.157460 Sep 25 00:59:32.801779 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 236/467] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:32 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 00:59:33.665446 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:33.666079 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:33.672926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:33.673440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:33.673873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:33.674559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:33.675006 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:33.675426 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:33.806495 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:33.814496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c50cf9b7-81cd-48b4-b2df-d15334ab6165 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1680 microversion: 2.1 time: 0.153046 Sep 25 00:59:33.815051 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 237/468] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:33 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1680 bytes in 154 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:59:33.826356 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:33.826998 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:33.833759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:33.834274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:33.834705 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:33.835420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:33.835907 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:33.836323 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:33.968400 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:33.976946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:33.978412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:33.979599 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-63b185e9-6919-4921-a818-300c5088522c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.156797 Sep 25 00:59:33.980097 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 238/469] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:33 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 158 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:34.841000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:34.841693 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:34.847951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:34.848422 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:34.848854 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:34.849578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:34.850009 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:34.850425 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:34.978092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:34.986839 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c4e81388-3868-4d2f-99bd-5948d79738ff tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1680 microversion: 2.1 time: 0.149407 Sep 25 00:59:34.987400 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 239/470] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:34 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1680 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:35.005417 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:35.006024 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:35.011812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:35.012293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:35.012728 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:35.013442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:35.013898 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:35.014320 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:35.149287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:35.158586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:35.160022 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:35.161168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-83936176-f5a6-4df7-8fdf-e16c2c97d579 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.159472 Sep 25 00:59:35.161748 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 240/471] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:35 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:36.010932 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:36.011759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:36.018200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:36.018616 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:36.019055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:36.019762 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:36.020216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:36.020754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:36.166629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:36.174566 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5c942a0a-c66a-44b4-aae6-2d6e14b93be8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1680 microversion: 2.1 time: 0.167649 Sep 25 00:59:36.175124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 241/472] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:36 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1680 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:59:36.188271 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:36.188880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:36.194892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:36.195310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:36.195741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:36.196424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:36.196875 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:36.197327 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:36.331433 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:36.340291 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:36.341980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:36.343149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b3691510-8434-49b4-b730-1844df84c44e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1810 microversion: 2.20 time: 0.159371 Sep 25 00:59:36.349446 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 242/473] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:36 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1810 bytes in 166 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:36.447307 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-27d55c17-733a-4545-8016-79970d254fda tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 4.208289 Sep 25 00:59:36.448108 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 232/474] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 00:59:32 2022] POST /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments => generated 194 bytes in 4210 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 00:59:37.198119 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:37.198844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:37.206081 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:37.206660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:37.207172 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:37.207949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:37.208496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:37.208992 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:37.290059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:37.298337 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a2a07003-150b-4071-aa4b-6fe06bbffe56 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1613 microversion: 2.1 time: 0.104058 Sep 25 00:59:37.298926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 243/475] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:37 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1613 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:37.371311 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:37.372015 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:37.380284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:37.380779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:37.381215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:37.381942 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:37.382396 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:37.382914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:37.525927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:37.536567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:37.538249 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:37.539702 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7c7e5fc7-dc70-443e-8801-4ed2602258c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1797 microversion: 2.20 time: 0.173380 Sep 25 00:59:37.540670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 233/476] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 00:59:37 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1797 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 00:59:37.828447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-0a9559dd-27f0-43f9-afc5-e19a7411643b req-dd591124-cd12-4d55-8c3a-7f07df8185b7 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "6892e391-59db-43ba-8843-ce34ade53390", "name": "network-vif-deleted", "tag": "36da5046-389c-4afa-9d14-41833f5b0db3"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:37.840318 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-0a9559dd-27f0-43f9-afc5-e19a7411643b req-dd591124-cd12-4d55-8c3a-7f07df8185b7 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:37.840803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-0a9559dd-27f0-43f9-afc5-e19a7411643b req-dd591124-cd12-4d55-8c3a-7f07df8185b7 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:37.841294 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-0a9559dd-27f0-43f9-afc5-e19a7411643b req-dd591124-cd12-4d55-8c3a-7f07df8185b7 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:37.853328 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-0a9559dd-27f0-43f9-afc5-e19a7411643b req-dd591124-cd12-4d55-8c3a-7f07df8185b7 service nova] Creating event network-vif-deleted:36da5046-389c-4afa-9d14-41833f5b0db3 for instance 6892e391-59db-43ba-8843-ce34ade53390 on hv-cinder-maste Sep 25 00:59:37.859717 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-0a9559dd-27f0-43f9-afc5-e19a7411643b req-dd591124-cd12-4d55-8c3a-7f07df8185b7 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035830 Sep 25 00:59:37.860332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 244/477] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:59:37 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:38.323259 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:38.324135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:38.331670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:38.332156 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:38.332590 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:38.333313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:38.333767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:38.334187 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:38.406822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:38.416181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a9af2fe6-a971-4943-aa4f-0f5672400910 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1613 microversion: 2.1 time: 0.096922 Sep 25 00:59:38.420691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 234/478] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:38 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1613 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:39.441691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:39.442391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:39.449099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:39.449623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:39.450060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:39.450774 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:39.451228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:39.451645 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:39.519429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:39.527704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9019b251-5391-4c48-9b94-ff4601f5bcaf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 200 len: 1613 microversion: 2.1 time: 0.091361 Sep 25 00:59:39.528316 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 245/479] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:39 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 1613 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:40.552820 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:40.553466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:40.560271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:40.560751 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:40.561223 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:40.561980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:40.562596 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:40.563023 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:40.577697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 HTTP/1.1 Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:59:40.579371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 00:59:40.581004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '6892e391-59db-43ba-8843-ce34ade53390'} Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:59:40.581838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 00:59:40.583037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:40.584116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:59:40.584116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6a6fb850-3f37-43de-be78-3df1f9204a3e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 404 len: 111 microversion: 2.1 time: 0.031928 Sep 25 00:59:40.584116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 235/480] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:40 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:59:40.589765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-570ba132-8671-4173-a237-3753f308d7c8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '.version_select at 0x7fe22db65820>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:40.611219 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-570ba132-8671-4173-a237-3753f308d7c8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-284669568" status: 202 len: 0 microversion: 2.1 time: 0.025146 Sep 25 00:59:40.612218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 246/481] 10.222.0.52 () {58 vars in 1218 bytes} [Sun Sep 25 00:59:40 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-284669568 => generated 0 bytes in 27 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 00:59:41.326301 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:41.327394 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:41.333932 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:41.334416 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:41.334844 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:41.335536 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:41.335981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:41.336410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:41.483678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lazy-loading 'services' on Instance uuid af139b53-2db6-43ea-9e69-27dfbb9304bc {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:59:41.605399 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e3b9d3b8-de10-4fd5-b8a1-c16fdce1a323 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments/28ae3ae1-26a0-462e-8a8f-d89670d28529" status: 202 len: 0 microversion: 2.1 time: 0.282767 Sep 25 00:59:41.606065 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 236/482] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 00:59:41 2022] DELETE /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments/28ae3ae1-26a0-462e-8a8f-d89670d28529 => generated 0 bytes in 284 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 00:59:41.615569 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:41.617927 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:41.627000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:41.627491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:41.627922 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:41.628704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:41.629327 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:41.629759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:41.660153 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-13ba486e-937e-422e-a931-086c8ed5d221 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048790 Sep 25 00:59:41.660730 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 247/483] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 00:59:41 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:42.673382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:42.676053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:42.682971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:42.683444 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:42.683909 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:42.684591 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:42.685047 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:42.685506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:42.717188 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6d3b8313-0f7f-409d-9d8f-5dac8423d48b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048588 Sep 25 00:59:42.718570 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 237/484] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 00:59:42 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:43.730633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:43.733795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:43.740830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:43.741240 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:43.741710 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:43.742427 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:43.742889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:43.743317 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:43.781582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-82a225b2-40fc-4289-9001-5291d1d1652c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054879 Sep 25 00:59:43.782159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 248/485] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 00:59:43 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:44.794379 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:44.797044 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:44.800060 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:44.804867 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:44.805386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:44.805827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:44.806523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:44.806980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:44.807405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:44.840010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5f1126b0-bbd3-4154-bb9b-39f4767b8d15 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.049645 Sep 25 00:59:44.840571 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 238/486] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 00:59:44 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments => generated 197 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:45.852589 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:45.855188 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:45.862473 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:45.862956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:45.863386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:45.864104 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:45.864554 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:45.864974 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:45.867453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:45.897218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0846bee8-3ee3-4dd7-ad9c-146ac3acc87f tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.048949 Sep 25 00:59:45.897805 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 249/487] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 00:59:45 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments => generated 197 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:46.910626 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:46.912836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:46.919832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:46.920980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:46.921594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:46.922387 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:46.922849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:46.923273 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:46.957675 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8e07bfaf-0efa-45db-bb2f-0b7f1b0958cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.050943 Sep 25 00:59:46.958455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 239/488] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 00:59:46 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:47.250377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'action', calling method: >, body: {"rescue": {"adminPass": "***"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:47.252646 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:47.259408 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:47.259912 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:47.260350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:47.261056 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:47.261560 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:47.261986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:47.287279 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lazy-loading 'flavor' on Instance uuid 621dfc45-048c-452f-86ca-ffdaeeb5f919 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:59:47.426690 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9ef7fa90-07fa-4309-9455-21b0c92d2ec3 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/action" status: 200 len: 32 microversion: 2.1 time: 0.180597 Sep 25 00:59:47.427297 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 250/489] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 00:59:47 2022] POST /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/action => generated 32 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 25 00:59:47.438253 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:47.438980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:47.445090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:47.445695 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:47.446130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:47.446830 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:47.447284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:47.447717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:47.595686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:47.605280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f40e447b-5bbb-4352-8f7b-60535cabe89d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.171940 Sep 25 00:59:47.606376 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 240/490] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:47 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:47.970861 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:47.973107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:47.979468 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:47.979937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:47.980378 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:47.981087 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:47.981571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:47.982092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:48.016699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-92876b1d-b8c2-41d1-9977-4c31d46799b3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.050136 Sep 25 00:59:48.017630 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 251/491] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 00:59:47 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc/os-volume_attachments => generated 25 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 25 00:59:48.087711 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:48.088358 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:48.095339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:48.095829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:48.096281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:48.096987 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:48.097473 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:48.097897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:48.127430 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 00:59:48.219646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7859b657-604d-499b-bdb4-bb79146294e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 204 len: 0 microversion: 2.1 time: 0.135349 Sep 25 00:59:48.220123 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 241/492] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 00:59:48 2022] DELETE /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 00:59:48.228608 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:48.229238 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:48.235963 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:48.236710 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:48.237181 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:48.237912 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:48.238364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:48.238788 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:48.385544 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:48.393816 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ebabdd03-ea2b-42e0-9ca8-8b8aefdebfb6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1679 microversion: 2.1 time: 0.168998 Sep 25 00:59:48.394392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 252/493] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:48 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1679 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:48.629210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:48.629920 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:48.635906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:48.636386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:48.636814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:48.637554 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:48.638000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:48.638424 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:48.776228 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:48.785129 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2154a78e-b45b-48c3-b755-bdbe0971f19f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.160048 Sep 25 00:59:48.785801 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 242/494] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:48 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 162 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:49.417997 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:49.418648 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:49.425382 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:49.425913 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:49.426385 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:49.427120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:49.427575 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:49.428020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:49.567658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:49.575560 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-76788ff9-addc-43de-9f44-621aac720a4c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1679 microversion: 2.1 time: 0.161569 Sep 25 00:59:49.576055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 253/495] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:49 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1679 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:49.810486 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:49.811708 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:49.819190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:49.819763 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:49.820202 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:49.820906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:49.821397 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:49.821817 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:49.959198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:49.967963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4a60a122-6451-4c05-ab5a-24678a87684c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.161446 Sep 25 00:59:49.968620 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 243/496] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:49 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:50.600309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:50.601005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:50.607014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:50.607518 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:50.607980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:50.608751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:50.609288 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:50.609742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:50.836177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:50.844560 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7f466cdf-d0ed-4759-99b3-c7696f03020a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1679 microversion: 2.1 time: 0.248721 Sep 25 00:59:50.845292 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 254/497] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:50 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1679 bytes in 250 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:50.994826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:50.995578 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:51.002395 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:51.002907 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:51.003328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:51.004041 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:51.004498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:51.004963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:51.158226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:51.162310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:51.168000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-04fb9238-d99d-4fbc-bea8-c016ba9bd7e6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.177694 Sep 25 00:59:51.168584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 244/498] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:50 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:51.431609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-8dfbc8ba-40ee-4072-bfcc-81e39450d8e8 req-41961b7a-1243-4335-b426-cc50dd1eb728 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "af139b53-2db6-43ea-9e69-27dfbb9304bc", "name": "network-vif-deleted", "tag": "0b0017e5-0afb-4334-8450-1d4303b0d473"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:51.450033 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8dfbc8ba-40ee-4072-bfcc-81e39450d8e8 req-41961b7a-1243-4335-b426-cc50dd1eb728 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:51.450845 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8dfbc8ba-40ee-4072-bfcc-81e39450d8e8 req-41961b7a-1243-4335-b426-cc50dd1eb728 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:51.451617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8dfbc8ba-40ee-4072-bfcc-81e39450d8e8 req-41961b7a-1243-4335-b426-cc50dd1eb728 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:51.474461 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-8dfbc8ba-40ee-4072-bfcc-81e39450d8e8 req-41961b7a-1243-4335-b426-cc50dd1eb728 service nova] Creating event network-vif-deleted:0b0017e5-0afb-4334-8450-1d4303b0d473 for instance af139b53-2db6-43ea-9e69-27dfbb9304bc on hv-cinder-maste Sep 25 00:59:51.495683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-8dfbc8ba-40ee-4072-bfcc-81e39450d8e8 req-41961b7a-1243-4335-b426-cc50dd1eb728 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.067559 Sep 25 00:59:51.496993 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 255/499] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:59:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:51.871687 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:51.873681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:51.884970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:51.885653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:51.886246 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:51.887326 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:51.888047 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:51.889650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:51.934218 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:51.934688 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:51.935117 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:51.936086 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lazy-loading 'fault' on Instance uuid af139b53-2db6-43ea-9e69-27dfbb9304bc {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 00:59:51.992523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:52.001865 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7b33e238-b323-448e-8900-c9ae0c279a57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 200 len: 1331 microversion: 2.1 time: 0.136451 Sep 25 00:59:52.002899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 245/500] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:51 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 1331 bytes in 138 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:52.195128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:52.196173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:52.205854 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:52.206669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:52.207409 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:52.208202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:52.208740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:52.209529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:52.396188 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:52.411952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-90d3abfa-5f05-470f-ae8a-6b33d750c220 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.223014 Sep 25 00:59:52.412542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 256/501] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:52 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:53.025131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:53.025800 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:53.033391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:53.033892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:53.034325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:53.035030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:53.035489 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:53.035912 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:53.050289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc HTTP/1.1 Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 00:59:53.052062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 00:59:53.053637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'af139b53-2db6-43ea-9e69-27dfbb9304bc'} Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 00:59:53.054537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 00:59:53.056078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 00:59:53.057470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 00:59:53.057470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bf0ffdf6-233f-4917-9810-19cd239ab409 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 404 len: 111 microversion: 2.1 time: 0.032320 Sep 25 00:59:53.057470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 246/502] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:53 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 00:59:53.062655 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-71cf467c-2867-4bd4-861b-08f5922b051b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '.version_select at 0x7fe22da78dc0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:53.083095 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-71cf467c-2867-4bd4-861b-08f5922b051b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1746086312" status: 202 len: 0 microversion: 2.1 time: 0.024153 Sep 25 00:59:53.083676 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 257/503] 10.222.0.52 () {58 vars in 1221 bytes} [Sun Sep 25 00:59:53 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1746086312 => generated 0 bytes in 25 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 00:59:53.460268 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:53.461002 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:53.469277 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:53.469981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:53.470420 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:53.471122 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:53.471578 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:53.471999 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:53.625679 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:53.634378 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-125f1740-e79e-420c-912c-b25adf801362 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.178360 Sep 25 00:59:53.635056 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 247/504] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:53 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:53.997790 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3c1b930f-30f1-424f-8e05-35fd1da93bfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: .version_select at 0x7fe22d959430>, body: {"keypair": {"name": "tempest-keypair-127083720", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBJSRfTo4jHaYF0v4OowxkcfhhdCCjD1lDHWZ54+9vjYjfvgQ/2CeZ1n/KvkQ5/ljgCns0Nu2hW0c4Zja09UhvnNLezOwmgj/PXyJVNgtv4Mv5/QkOwAKR4+UB5NJxJb21g=="}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:54.014669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-3c1b930f-30f1-424f-8e05-35fd1da93bfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:59:54.019881 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-3c1b930f-30f1-424f-8e05-35fd1da93bfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for user 96386a16f0b54b7c94d32bd948caf118 and project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:59:54.043868 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3c1b930f-30f1-424f-8e05-35fd1da93bfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.049876 Sep 25 00:59:54.044473 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 258/505] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 00:59:53 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:54.660486 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:54.661212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:54.668710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:54.669213 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:54.669750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:54.671643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:54.672022 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:54.672452 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:54.811783 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:54.820419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6b2c678e-36cf-4453-832e-e06818454d1a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.163922 Sep 25 00:59:54.820986 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 248/506] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:54 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:55.652688 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-563830042", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5dc49b54-686c-4619-ad4c-65c72c26fd31"}], "security_groups": [{"name": "tempest-securitygroup--76657188"}], "key_name": "tempest-keypair-127083720", "user_data": "}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:55.662791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=73,nova_api:DELETE=2,nova_api:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 00:59:55.766785 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] validate_networks() for [('5dc49b54-686c-4619-ad4c-65c72c26fd31', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 00:59:55.844152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:55.845057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:55.851956 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:55.852501 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:55.852897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:55.853709 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:55.854167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:55.854606 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:55.994548 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:56.003507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-57638cae-f593-4810-945e-d4f515578cfd tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.163174 Sep 25 00:59:56.004084 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 249/507] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:55 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:59:56.163689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 00:59:56.164182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 00:59:56.164581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 00:59:56.164989 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 00:59:56.165422 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 00:59:56.175279 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:56.175758 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:56.176187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:56.176906 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:56.177385 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:56.177806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:56.196582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 00:59:56.200803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for user 96386a16f0b54b7c94d32bd948caf118 and project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 00:59:56.213156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 00:59:56.215637 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 00:59:56.266924 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fbac6f5c-439e-4d75-8b0d-54cdf69cf6b1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 400 microversion: 2.1 time: 0.619877 Sep 25 00:59:56.267716 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 259/508] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 00:59:55 2022] POST /compute/v2.1/servers => generated 400 bytes in 622 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 00:59:56.279065 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-56590d82-1de7-4c26-8111-35ca59b0db7b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:56.280400 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-56590d82-1de7-4c26-8111-35ca59b0db7b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:56.335996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-56590d82-1de7-4c26-8111-35ca59b0db7b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:56.343279 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-56590d82-1de7-4c26-8111-35ca59b0db7b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1240 microversion: 2.1 time: 0.068072 Sep 25 00:59:56.344082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 250/509] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:56 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1240 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:59:57.028060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:57.028699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:57.035680 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:57.036159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:57.036593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:57.037321 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:57.037772 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:57.038194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:57.183789 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:57.193558 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9efaeadd-a22c-48fb-976a-71d016171732 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.169200 Sep 25 00:59:57.208509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 260/510] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:57 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 00:59:57.367515 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:57.368066 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:57.376691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:57.377182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:57.377661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:57.378358 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:57.378813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:57.379274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:57.445190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:57.455337 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ac2d79e8-71dd-43ff-8e53-4f400bc82afb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1292 microversion: 2.1 time: 0.091777 Sep 25 00:59:57.455915 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 251/511] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:57 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1292 bytes in 93 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:58.219146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:58.219886 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:58.227466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:58.227970 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:58.228412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:58.229118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:58.229621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:58.230034 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:58.427217 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:58.437336 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-aedc885b-d415-4d00-9429-6e0d8896bb8b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.224201 Sep 25 00:59:58.438184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 261/512] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:58 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:58.480986 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:58.482507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:58.490369 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:58.491137 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:58.491727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:58.492849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:58.493593 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:58.494182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:58.707495 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:58.715532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5e2c53ee-5923-4338-bff9-a5e7b42c6c97 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.238441 Sep 25 00:59:58.716121 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 252/513] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:58 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 240 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:59.352215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-34db1abd-8128-4222-926c-b163923b52d0 req-8cd8f09b-95a7-43af-95d4-ec2676d21196 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e53662f0-18c6-40e1-99da-07e0065395e1", "tag": "dc9429b9-a8b5-45cb-94af-f68e5fc7099c"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 00:59:59.365469 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-34db1abd-8128-4222-926c-b163923b52d0 req-8cd8f09b-95a7-43af-95d4-ec2676d21196 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:59.365959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-34db1abd-8128-4222-926c-b163923b52d0 req-8cd8f09b-95a7-43af-95d4-ec2676d21196 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:59.366429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-34db1abd-8128-4222-926c-b163923b52d0 req-8cd8f09b-95a7-43af-95d4-ec2676d21196 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:59.379647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-34db1abd-8128-4222-926c-b163923b52d0 req-8cd8f09b-95a7-43af-95d4-ec2676d21196 service nova] Creating event network-changed:dc9429b9-a8b5-45cb-94af-f68e5fc7099c for instance e53662f0-18c6-40e1-99da-07e0065395e1 on hv-cinder-maste Sep 25 00:59:59.388499 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-34db1abd-8128-4222-926c-b163923b52d0 req-8cd8f09b-95a7-43af-95d4-ec2676d21196 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036306 Sep 25 00:59:59.388499 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 262/514] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 00:59:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 00:59:59.469045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:59.469708 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:59.475836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:59.476324 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:59.476766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:59.477492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:59.477948 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:59.478372 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:59.613541 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:59.622070 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-58fb2544-b3d4-418f-abef-6ad901e873c2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.157549 Sep 25 00:59:59.622650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 253/515] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:59 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 159 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 00:59:59.740329 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 00:59:59.741017 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 00:59:59.747210 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:59.747720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:59.748320 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:59.749200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 00:59:59.749678 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 00:59:59.750102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 00:59:59.934430 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 00:59:59.943054 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-53489dd3-f02f-4bde-9184-7481c00a8b49 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.207263 Sep 25 00:59:59.943683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 263/516] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 00:59:59 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:00.645674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:00.646291 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:00.652814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:00.653334 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:00.653779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:00.654644 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:00.655111 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:00.655541 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:00.803034 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:00.814940 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-dce6f902-94af-4c6f-99ab-1502af16d252 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.172225 Sep 25 01:00:00.815766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 254/517] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:00 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:00.968618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:00.969525 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:00.976334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:00.977119 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:00.977747 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:00.978512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:00.979012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:00.979441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:01.123481 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:01.133365 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ca2fe766-4b5e-4390-9954-610d82f8453a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.170378 Sep 25 01:00:01.133933 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 264/518] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:00 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:01.843373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:01.843925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:01.850591 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:01.851093 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:01.851537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:01.852248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:01.853120 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:01.853120 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:01.994165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:02.004452 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ebda9847-684e-44b0-9253-cd58ba947cd5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.165369 Sep 25 01:00:02.005713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 255/519] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:01 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:02.155879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Action: 'action', calling method: >, body: {"reboot": {"type": "SOFT"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:02.156449 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:02.156951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:02.158699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:02.164111 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:02.164682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:02.165370 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:02.167462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:02.167591 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:02.168118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:02.168257 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:02.168700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:02.168820 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:02.169575 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:02.171054 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:02.171921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:02.267324 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a0da8135-9f6a-4660-8523-116e08dbd724 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "POST /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/action" status: 202 len: 0 microversion: 2.1 time: 0.118345 Sep 25 01:00:02.269033 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 265/520] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:00:02 2022] POST /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/action => generated 0 bytes in 121 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:00:02.278481 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:02.280132 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:02.289309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:02.289812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:02.290248 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:02.291121 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:02.291616 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:02.292044 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:02.347032 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:02.355870 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d0c711d2-979e-4a06-a4ae-725154aa25a1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.203760 Sep 25 01:00:02.356459 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 256/521] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:02 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:02.545403 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:02.548284 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:02.560295 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e1ec3382-80df-4616-b681-f77c0b77da98 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1751 microversion: 2.1 time: 0.286589 Sep 25 01:00:02.561750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 266/522] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:02 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1751 bytes in 289 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:03.027998 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:03.028694 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:03.035400 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:03.035891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:03.036327 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:03.037024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:03.037605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:03.037940 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:03.189544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:03.198299 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-87401844-7ca2-4b24-b086-412765604772 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.174083 Sep 25 01:00:03.199020 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 257/523] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:03 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:03.380124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:03.380735 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:03.392983 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:03.393357 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:03.393921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:03.394495 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:03.394952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:03.395381 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:03.533958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:03.541677 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9e42f3d7-737d-48e9-b833-bb9b433479e2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.165388 Sep 25 01:00:03.542236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 267/524] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:03 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:03.584014 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:03.584626 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:03.590659 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:03.591142 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:03.592235 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:03.592931 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:03.593430 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:03.593854 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:03.746610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:03.756105 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1aa28fdb-654c-40ff-a318-564653cea93c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.175921 Sep 25 01:00:03.756688 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 258/525] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:03 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:04.223062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:04.223587 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:04.229659 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:04.230143 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:04.230580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:04.231304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:04.231766 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:04.232194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:04.378709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:04.387503 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-76f32239-9b63-4674-92d9-73e30b6d30da tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.168337 Sep 25 01:00:04.388277 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 268/526] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:04 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:04.564411 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:04.565039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:04.571440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:04.571933 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:04.572375 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:04.573073 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:04.573572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:04.573997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:04.719492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:04.727504 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-81c6526e-5dd3-4b3c-a3ca-64c19e2715af tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.167277 Sep 25 01:00:04.728079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 259/527] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:04 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:04.780994 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:04.781627 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:04.788455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:04.789063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:04.789618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:04.790310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:04.790786 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:04.791204 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:04.943028 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:04.952325 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-12c8335d-cd09-47dd-9b5f-e792601e349a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.176009 Sep 25 01:00:04.952918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 269/528] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:04 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:05.413182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:05.414102 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:05.420640 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:05.421141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:05.421609 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:05.422305 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:05.422765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:05.423191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:05.566115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:05.574747 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c9794918-7303-4005-837e-98f3e99c498e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1737 microversion: 2.1 time: 0.165364 Sep 25 01:00:05.575323 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 260/529] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:05 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1737 bytes in 167 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:05.750861 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:05.751480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:05.757936 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:05.758419 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:05.758850 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:05.759542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:05.759987 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:05.760470 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:05.889310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-f9e5032d-b0e2-4a6d-b12d-0bdd4be7ac70 req-ef9b3b17-dcb6-4993-980b-576336359d3e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e53662f0-18c6-40e1-99da-07e0065395e1", "name": "network-vif-plugged", "status": "completed", "tag": "dc9429b9-a8b5-45cb-94af-f68e5fc7099c"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:05.900596 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-f9e5032d-b0e2-4a6d-b12d-0bdd4be7ac70 req-ef9b3b17-dcb6-4993-980b-576336359d3e service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:05.901103 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-f9e5032d-b0e2-4a6d-b12d-0bdd4be7ac70 req-ef9b3b17-dcb6-4993-980b-576336359d3e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:05.902263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-f9e5032d-b0e2-4a6d-b12d-0bdd4be7ac70 req-ef9b3b17-dcb6-4993-980b-576336359d3e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:05.906235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:05.915679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-70713946-d68a-422a-b45c-94749c1c70b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.168789 Sep 25 01:00:05.916198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-f9e5032d-b0e2-4a6d-b12d-0bdd4be7ac70 req-ef9b3b17-dcb6-4993-980b-576336359d3e service nova] Creating event network-vif-plugged:dc9429b9-a8b5-45cb-94af-f68e5fc7099c for instance e53662f0-18c6-40e1-99da-07e0065395e1 on hv-cinder-maste Sep 25 01:00:05.917328 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 270/530] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:05 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:05.922357 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-f9e5032d-b0e2-4a6d-b12d-0bdd4be7ac70 req-ef9b3b17-dcb6-4993-980b-576336359d3e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036802 Sep 25 01:00:05.923179 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 261/531] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:05 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:00:05.976227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:05.976828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:05.983957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:05.984435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:05.984865 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:05.985616 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:05.986068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:05.986740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:06.148947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:06.158136 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c18bd2a3-a941-43af-ac70-e69d2337a06e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.185877 Sep 25 01:00:06.158650 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 271/532] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:05 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:06.599510 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:06.600261 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:06.602531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:06.609919 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:06.610412 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:06.610854 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:06.611865 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:06.612328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:06.612769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:06.759667 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:06.768284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-420c4ce8-7348-46d0-b959-b23179584a82 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1717 microversion: 2.1 time: 0.173241 Sep 25 01:00:06.768900 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 262/533] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:06 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1717 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:06.882566 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:06.883196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:06.884933 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:06.885619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=10 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:06.891628 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:06.892117 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:06.892558 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:06.893271 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:06.893738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:06.894193 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:06.942667 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:06.943183 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:06.949334 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:06.949817 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:06.950251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:06.951153 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:06.951786 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:06.952207 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:06.972041 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:07.025137 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] HTTP exception thrown: Cannot 'detach_volume' instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 while it is in vm_state rescued Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Request method failure captured: Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: DELETE /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments/0ff9470c-98a0-4477-980a-1fb9e8d393c5 HTTP/1.1 Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cb8QFEWahSITkrpCBqxYYBD9zi_oLSAVWAk9whdwsp48CleshPfYnV97ECBsJa63kh6to7hObTL2AlKDs6Ggck8j1eD8_mIGQLRF0aMsS9KlL5zrlzoseQDGerOSLOpzlzHV2LyCeRa0pLQWwIucZcMEZw7qNiXQLyFloWXCSSLWQ Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 28568584195d475fbc52d76185e09044 Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:00:07.026938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_28568584195d475fbc52d76185e09044", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/28568584195d475fbc52d76185e09044"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}] Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 28568584195d475fbc52d76185e09044 Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 8eabe2a74a604c47bc2e5c88b960dcb6 Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:00:07.028520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Cannot 'detach_volume' instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 while it is in vm_state rescued Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'server_id': '621dfc45-048c-452f-86ca-ffdaeeb5f919', 'id': '0ff9470c-98a0-4477-980a-1fb9e8d393c5'} Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 565, in delete Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi self.compute_api.detach_volume(context, instance, volume) Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 148, in inner Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceInvalidState( Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceInvalidState: Instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 in vm_state rescued. Cannot detach_volume while the instance is in this state. Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:00:07.029917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 573, in delete Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi common.raise_http_conflict_for_instance_invalid_state(state_error, Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 347, in raise_http_conflict_for_instance_invalid_state Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise webob.exc.HTTPConflict(explanation=msg) Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPConflict: Cannot 'detach_volume' instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 while it is in vm_state rescued Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Cannot 'detach_volume' instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 while it is in vm_state rescued Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:07.031579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Returning 409 to user: Cannot 'detach_volume' instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 while it is in vm_state rescued {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:00:07.032771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1b9f5fd7-5941-46a6-988d-dec31d6791e4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "DELETE /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments/0ff9470c-98a0-4477-980a-1fb9e8d393c5" status: 409 len: 152 microversion: 2.1 time: 0.152956 Sep 25 01:00:07.033230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 272/534] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:00:06 2022] DELETE /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments/0ff9470c-98a0-4477-980a-1fb9e8d393c5 => generated 152 bytes in 155 msecs (HTTP/1.1 409) 9 headers in 378 bytes (1 switches on core 0) Sep 25 01:00:07.041485 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Action: 'action', calling method: >, body: {"unrescue": {}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:07.042022 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:07.049974 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:07.050357 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:07.050763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:07.051472 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:07.051931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:07.052362 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:07.088891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:07.097979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d4659c1e-7a66-4545-8602-64c528c8443a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.159161 Sep 25 01:00:07.098567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 263/535] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:06 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:07.128339 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-19aa306e-dc76-4740-a8ae-add23857e794 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "POST /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/action" status: 202 len: 0 microversion: 2.1 time: 0.090495 Sep 25 01:00:07.128966 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 273/536] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:00:07 2022] POST /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/action => generated 0 bytes in 92 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 01:00:07.136985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:07.137631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:07.143591 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:07.144077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:07.144521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:07.145193 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:07.145683 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:07.146106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:07.182452 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:07.183149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:07.189758 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:07.190250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:07.190692 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:07.191405 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:07.191875 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:07.192300 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:07.284344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:07.292886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-df4058e4-ee69-441a-8113-dc731c07fda6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1725 microversion: 2.1 time: 0.159617 Sep 25 01:00:07.293516 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 264/537] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:07 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1725 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:07.342740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:07.351167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-10d27dc9-e6f4-43f0-b2d7-149b7ecbf703 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.172888 Sep 25 01:00:07.351731 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 274/538] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:07 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:08.122018 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:08.122953 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:08.129504 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:08.129996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:08.130437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:08.131173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:08.131630 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:08.132055 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:08.269839 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:08.277599 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2d046635-aff5-4587-9052-f9417cea63a2 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1364 microversion: 2.1 time: 0.159304 Sep 25 01:00:08.278087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 265/539] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:08 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1364 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:08.316930 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:08.317563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:08.323764 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:08.324255 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:08.324692 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:08.325458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:08.326604 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:08.327098 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:08.376132 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:08.376903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:08.382941 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:08.383417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:08.383851 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:08.384550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:08.384987 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:08.385456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:08.472009 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:08.480807 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-415c88a7-be49-49ac-9b46-6925d315fc0c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1725 microversion: 2.1 time: 0.167621 Sep 25 01:00:08.481958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 275/540] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:08 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1725 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:08.561646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:08.570692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-88f24457-8744-444c-b978-c73af88cd0a7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.199184 Sep 25 01:00:08.571286 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 266/541] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:08 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:09.300953 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:09.301487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:09.307546 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:09.308015 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:09.308443 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:09.309130 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:09.309611 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:09.310788 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:09.448271 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:09.457101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e6b6165f-2654-4a9d-b7e5-f21ca0d17141 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1554 microversion: 2.1 time: 0.160069 Sep 25 01:00:09.457708 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 276/542] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:09 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1554 bytes in 161 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:09.479745 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:09.480263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:09.487773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:09.488292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:09.488760 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:09.489518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:09.489975 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:09.490404 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:09.504442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:09.505058 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:09.511078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:09.511568 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:09.512006 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:09.512705 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:09.513747 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:09.514175 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:09.579378 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-df82d855-f518-4b00-bdaa-b0346f6c74a0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.103428 Sep 25 01:00:09.580001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 267/543] 10.222.0.52 () {58 vars in 1275 bytes} [Sun Sep 25 01:00:09 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-interface => generated 285 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:09.595583 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:09.596522 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:09.603192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:09.603818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:09.604347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:09.605192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:09.605793 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:09.606612 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:09.664673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:09.674143 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cc571b58-2be8-4f0b-a896-e9363e21e176 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1725 microversion: 2.1 time: 0.173376 Sep 25 01:00:09.674740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 277/544] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:09 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1725 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:09.895484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:09.904569 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1cd751eb-973c-4679-b81e-93acd9bc08b5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.312715 Sep 25 01:00:09.905146 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 268/545] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:09 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:10.649049 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-5b6f9ab8-2539-40ab-ab29-c8afa0ce7ee1 req-04b94721-9852-41d3-8704-848bf8fefa34 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "e53662f0-18c6-40e1-99da-07e0065395e1", "tag": "dc9429b9-a8b5-45cb-94af-f68e5fc7099c"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:10.663233 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-5b6f9ab8-2539-40ab-ab29-c8afa0ce7ee1 req-04b94721-9852-41d3-8704-848bf8fefa34 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:10.663728 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-5b6f9ab8-2539-40ab-ab29-c8afa0ce7ee1 req-04b94721-9852-41d3-8704-848bf8fefa34 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:10.664160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-5b6f9ab8-2539-40ab-ab29-c8afa0ce7ee1 req-04b94721-9852-41d3-8704-848bf8fefa34 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:10.675768 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-5b6f9ab8-2539-40ab-ab29-c8afa0ce7ee1 req-04b94721-9852-41d3-8704-848bf8fefa34 service nova] Creating event network-changed:dc9429b9-a8b5-45cb-94af-f68e5fc7099c for instance e53662f0-18c6-40e1-99da-07e0065395e1 on hv-cinder-maste Sep 25 01:00:10.680748 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-5b6f9ab8-2539-40ab-ab29-c8afa0ce7ee1 req-04b94721-9852-41d3-8704-848bf8fefa34 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.035326 Sep 25 01:00:10.681371 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 278/546] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:10.703931 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:10.705075 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:10.715501 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:10.716872 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:10.717901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:10.719275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:10.719893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:10.720740 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:10.868319 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:10.877238 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7c8b4954-4927-4c13-8fad-c9e7b1cff2a6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1725 microversion: 2.1 time: 0.181035 Sep 25 01:00:10.877852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 269/547] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:10 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1725 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:10.932562 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:10.933520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:10.941793 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:10.942634 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:10.943301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:10.944179 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:10.944809 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:10.945415 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:11.141962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:11.150332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8af53ac2-7865-41b8-8433-d8db2cadf13e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.221926 Sep 25 01:00:11.150911 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 279/548] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:10 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:11.900096 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:11.901380 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:11.908893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:11.909427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:11.909871 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:11.910585 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:11.911046 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:11.911476 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:12.054441 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:12.063839 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4b70618c-cb72-4466-974b-a08851189781 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1731 microversion: 2.1 time: 0.167645 Sep 25 01:00:12.064515 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 270/549] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:11 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1731 bytes in 169 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:12.086815 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:12.087330 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:12.093344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:12.093828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:12.094265 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:12.094963 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:12.095420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:12.095842 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:12.175194 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:12.175807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:12.178494 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-56832279-4ed3-47ca-b9fe-64aefe558243 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.095494 Sep 25 01:00:12.179194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 280/550] 10.222.0.52 () {58 vars in 1275 bytes} [Sun Sep 25 01:00:12 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-interface => generated 285 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:12.182597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:12.183010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:12.183469 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:12.184226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:12.184637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:12.187518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:12.333851 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:12.342930 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-cae1ef54-1962-4350-9129-af232b4ddf9b tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.171864 Sep 25 01:00:12.343793 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 271/551] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:12 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:13.100631 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-1ee43270-93cd-4151-918b-b8ad9741400d req-aa5e09bb-b3ce-43fe-bbfd-28f631789ff8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "621dfc45-048c-452f-86ca-ffdaeeb5f919", "tag": "a554103e-40f9-42b8-b04a-4fc7d8888240"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:13.112844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1ee43270-93cd-4151-918b-b8ad9741400d req-aa5e09bb-b3ce-43fe-bbfd-28f631789ff8 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:13.113455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1ee43270-93cd-4151-918b-b8ad9741400d req-aa5e09bb-b3ce-43fe-bbfd-28f631789ff8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:13.113886 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1ee43270-93cd-4151-918b-b8ad9741400d req-aa5e09bb-b3ce-43fe-bbfd-28f631789ff8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:13.126520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-1ee43270-93cd-4151-918b-b8ad9741400d req-aa5e09bb-b3ce-43fe-bbfd-28f631789ff8 service nova] Creating event network-changed:a554103e-40f9-42b8-b04a-4fc7d8888240 for instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 on hv-cinder-maste Sep 25 01:00:13.132304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-1ee43270-93cd-4151-918b-b8ad9741400d req-aa5e09bb-b3ce-43fe-bbfd-28f631789ff8 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.036604 Sep 25 01:00:13.132905 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 281/552] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:13 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:13.367433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:13.368967 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:13.375678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:13.376164 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:13.376814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:13.377561 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:13.378024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:13.378702 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:13.541501 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:13.551282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-13060e9d-149b-46a7-ba8e-7d3228f4b071 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.188137 Sep 25 01:00:13.551961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 272/553] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:13 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:14.576187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:14.576980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:14.583143 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:14.583617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:14.584046 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:14.584729 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:14.585178 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:14.585629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:14.730634 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:14.739512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3a17d894-90e8-4959-b5df-3b0b330d8673 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.168173 Sep 25 01:00:14.740004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 282/554] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:14 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:15.146404 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-29c7bd72-4927-4ed1-8259-d088c29ce92d req-d2be295c-ac52-4fdd-a749-792c03a8cb8f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "621dfc45-048c-452f-86ca-ffdaeeb5f919", "tag": "a554103e-40f9-42b8-b04a-4fc7d8888240"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:15.151646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=79,nova_api:UPDATE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:15.158266 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-29c7bd72-4927-4ed1-8259-d088c29ce92d req-d2be295c-ac52-4fdd-a749-792c03a8cb8f service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:15.158747 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-29c7bd72-4927-4ed1-8259-d088c29ce92d req-d2be295c-ac52-4fdd-a749-792c03a8cb8f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:15.159182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-29c7bd72-4927-4ed1-8259-d088c29ce92d req-d2be295c-ac52-4fdd-a749-792c03a8cb8f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:15.171471 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-29c7bd72-4927-4ed1-8259-d088c29ce92d req-d2be295c-ac52-4fdd-a749-792c03a8cb8f service nova] Creating event network-changed:a554103e-40f9-42b8-b04a-4fc7d8888240 for instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 on hv-cinder-maste Sep 25 01:00:15.176627 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-29c7bd72-4927-4ed1-8259-d088c29ce92d req-d2be295c-ac52-4fdd-a749-792c03a8cb8f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.033934 Sep 25 01:00:15.177219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 273/555] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:15.763056 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:15.763674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:15.770679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:15.771722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:15.772323 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:15.773487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:15.774071 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:15.774502 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:15.934601 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:15.944209 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-734e94b5-3007-470c-a9c7-6ac72269f227 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.184925 Sep 25 01:00:15.944834 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 283/556] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:15 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:15.947902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:15.948590 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:15.955948 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:15.956516 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:15.957018 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:15.957835 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:15.958352 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:15.958840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:15.990977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-fde13d48-aaf5-4226-b5f4-e5387d0505a8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments/1cb3dd71-9b2f-4d99-bda4-350239151043" status: 200 len: 194 microversion: 2.20 time: 0.048467 Sep 25 01:00:15.991473 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 274/557] 10.222.0.52 () {60 vars in 1454 bytes} [Sun Sep 25 01:00:15 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments/1cb3dd71-9b2f-4d99-bda4-350239151043 => generated 194 bytes in 50 msecs (HTTP/1.1 200) 9 headers in 359 bytes (2 switches on core 0) Sep 25 01:00:16.064300 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:16.064801 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:16.071228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:16.071703 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:16.072138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:16.072828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:16.073314 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:16.073770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:16.077226 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:16.162083 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lazy-loading 'services' on Instance uuid 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:00:16.279787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-be1d5d58-2336-4164-8b70-93e85edddcd0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "DELETE /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments/1cb3dd71-9b2f-4d99-bda4-350239151043" status: 202 len: 0 microversion: 2.20 time: 0.218714 Sep 25 01:00:16.280465 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 284/558] 10.222.0.52 () {60 vars in 1457 bytes} [Sun Sep 25 01:00:16 2022] DELETE /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments/1cb3dd71-9b2f-4d99-bda4-350239151043 => generated 0 bytes in 221 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:00:16.290086 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:16.292528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:16.300490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:16.301013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:16.301505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:16.302241 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:16.302809 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:16.303243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:16.337208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1e08c0cd-e7b3-40f7-bdac-23ddb2add5a7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments" status: 200 len: 197 microversion: 2.20 time: 0.051621 Sep 25 01:00:16.338049 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 275/559] 10.222.0.52 () {60 vars in 1343 bytes} [Sun Sep 25 01:00:16 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments => generated 197 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 01:00:16.968756 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:16.969473 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:16.975831 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:16.976310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:16.976760 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:16.977462 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:16.977904 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:16.978327 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:17.128976 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:17.138414 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a983320c-56a0-4062-8757-ae351f3ec453 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.173277 Sep 25 01:00:17.138995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 285/560] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:16 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 175 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:17.351702 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:17.353967 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:17.361615 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:17.362755 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:17.362755 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:17.363206 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:17.363715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:17.364141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:17.397608 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a995b78c-6bd3-4abd-aa29-f1fb2c532ba3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.049829 Sep 25 01:00:17.398098 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 276/561] 10.222.0.52 () {60 vars in 1343 bytes} [Sun Sep 25 01:00:17 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9/os-volume_attachments => generated 25 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:17.463754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:17.464365 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:17.471126 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:17.471629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:17.472059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:17.472742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:17.473275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:17.473715 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:17.500466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:00:17.578514 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-417b3959-e8c0-4ec6-8a66-addd6d3e96e8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "DELETE /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 204 len: 0 microversion: 2.20 time: 0.118327 Sep 25 01:00:17.579383 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 286/562] 10.222.0.52 () {60 vars in 1280 bytes} [Sun Sep 25 01:00:17 2022] DELETE /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 25 01:00:17.588271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:17.588885 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:17.595890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:17.596384 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:17.596854 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:17.597609 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:17.598067 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:17.598493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:17.735376 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:17.744854 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:17.746355 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:17.748490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-10d9e93f-ae7a-4765-832a-da2c6d80b627 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.163785 Sep 25 01:00:17.749049 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 277/563] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:17 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:18.162608 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:18.163301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:18.170180 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:18.170723 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:18.171341 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:18.171856 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:18.172307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:18.172732 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:18.322160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:18.331426 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-482509ca-8b01-46f5-94fd-5c633290fbc5 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.172923 Sep 25 01:00:18.331999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 287/564] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:18 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:18.774519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:18.775157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:18.782631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:18.783119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:18.783565 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:18.784267 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:18.784722 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:18.785150 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:18.922138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:18.930309 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:18.931752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:18.932932 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6a76d55a-3d63-415d-adb7-f7a71fae7187 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.163412 Sep 25 01:00:18.933525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 278/565] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:18 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 165 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:19.355964 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:19.356613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:19.366825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:19.367310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:19.367746 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:19.368449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:19.368901 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:19.369373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:19.521744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:19.531851 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-442180b8-59f6-4ae6-b9b8-aa20075cb461 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1756 microversion: 2.1 time: 0.179732 Sep 25 01:00:19.532419 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 288/566] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:19 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1756 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:19.958843 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:19.959482 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:19.965995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:19.966480 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:19.966914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:19.967632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:19.968117 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:19.968542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:20.105507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:20.114038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:20.115469 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:20.116611 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0dbc6024-d96d-4faa-ae9d-683d5a9322da tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1725 microversion: 2.20 time: 0.161952 Sep 25 01:00:20.117172 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 279/567] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:19 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1725 bytes in 163 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:20.556092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:20.556717 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:20.563478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:20.563965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:20.564517 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:20.565128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:20.565701 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:20.566137 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:20.824752 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:20.833175 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4cbd2dff-43f3-4505-b497-b33d8a26b294 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1759 microversion: 2.1 time: 0.280918 Sep 25 01:00:20.833793 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 289/568] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:20 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1759 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:21.143766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:21.144385 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:21.151107 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:21.151588 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:21.152037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:21.152717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:21.153775 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:21.154513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:21.221339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:21.230115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:21.231769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:21.233179 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7c054c67-6be9-4ca3-86e3-41e5a3f66cfc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1658 microversion: 2.20 time: 0.093321 Sep 25 01:00:21.233781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 280/569] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:21 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1658 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:21.478536 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-b98c6e5f-bef5-49b7-849e-47adfb0c3f29 req-dc18fd2b-cdbe-4aa7-aaad-7be6423625e0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9", "name": "network-vif-deleted", "tag": "96168749-3e0f-462f-9aed-adacd4fb28c0"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:21.491664 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-b98c6e5f-bef5-49b7-849e-47adfb0c3f29 req-dc18fd2b-cdbe-4aa7-aaad-7be6423625e0 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:21.492151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-b98c6e5f-bef5-49b7-849e-47adfb0c3f29 req-dc18fd2b-cdbe-4aa7-aaad-7be6423625e0 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:21.492585 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-b98c6e5f-bef5-49b7-849e-47adfb0c3f29 req-dc18fd2b-cdbe-4aa7-aaad-7be6423625e0 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:21.505918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-b98c6e5f-bef5-49b7-849e-47adfb0c3f29 req-dc18fd2b-cdbe-4aa7-aaad-7be6423625e0 service nova] Creating event network-vif-deleted:96168749-3e0f-462f-9aed-adacd4fb28c0 for instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 on hv-cinder-maste Sep 25 01:00:21.511825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-b98c6e5f-bef5-49b7-849e-47adfb0c3f29 req-dc18fd2b-cdbe-4aa7-aaad-7be6423625e0 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.036696 Sep 25 01:00:21.513028 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 290/570] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:22.259357 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:22.260685 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:22.268298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:22.269830 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:22.270527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:22.271893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:22.277141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:22.279383 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:22.284644 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:22.316939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:22.317463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:22.318088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:22.318541 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lazy-loading 'fault' on Instance uuid 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:00:22.363864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:22.373923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:22.376041 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:22.377552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f3116f78-194b-40ed-931f-094a97a0f0a9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 200 len: 1372 microversion: 2.20 time: 0.122181 Sep 25 01:00:22.378204 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 281/571] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:22 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 1372 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:23.404456 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:23.405149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:23.411575 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:23.412061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:23.412499 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:23.413195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:23.413677 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:23.414101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:23.428572 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] HTTP exception thrown: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Request method failure captured: Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 HTTP/1.1 Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cgaG8JioxfKBR838yc2ImiVIZR2nzrQQb1CDqfoLEgRRIeq_T_1Jh4FQMpH3FQxKYM_tzvdc2MiB8ugmOEJ2-WAHkXrJ5-Zi0kqn7d2PpMRaT2dcf155Id7yyuHxFCRYc8q5ds3uX1ckQuFFhXY9EnUqrkQnQuqtXoLPq4JYZ5D78 Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Openstack-Nova-Api-Version: 2.20 Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:00:23.430250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f09655506e354c9d991f4df9a92f4086", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}] Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: dbe551f8c10c4111b4ad7380a6482564 Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:00:23.431783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9'} Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:00:23.432565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:00:23.433855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:00:23.435780 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Returning 404 to user: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:00:23.435780 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f338e58c-67e4-4d3c-b6e2-d863b7e0a4e2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 404 len: 111 microversion: 2.20 time: 0.031634 Sep 25 01:00:23.435780 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 291/572] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:23 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 111 bytes in 33 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 25 01:00:23.441605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bf0a48e9-ade4-47c2-a2d8-e646775331c0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '.version_select at 0x7fe22d855d30>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:23.461212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bf0a48e9-ade4-47c2-a2d8-e646775331c0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1654387516" status: 204 len: 0 microversion: 2.20 time: 0.023520 Sep 25 01:00:23.461822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 282/573] 10.222.0.52 () {60 vars in 1262 bytes} [Sun Sep 25 01:00:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1654387516 => generated 0 bytes in 25 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 25 01:00:24.131057 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9f8c02a1-6d6d-4140-920d-1f049c4c6da3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'create', calling method: .version_select at 0x7fe22da660d0>, body: {"keypair": {"name": "tempest-keypair-2072723538", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBNMx/6RgEmyQks09qMmab3+9NMFKtcOzZCIxARv1hVAlDolovo3EADpmTkgTPYQlD+7IHye1UMVGeTaZHyY4lVMAzLL3PDYUg0VqqmI74aAYsubbmuaG8wC5KOsKKFL42g=="}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:24.147371 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-9f8c02a1-6d6d-4140-920d-1f049c4c6da3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Getting quotas for project f09655506e354c9d991f4df9a92f4086. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:00:24.151213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-9f8c02a1-6d6d-4140-920d-1f049c4c6da3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Getting quotas for user dbe551f8c10c4111b4ad7380a6482564 and project f09655506e354c9d991f4df9a92f4086. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:00:24.174214 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9f8c02a1-6d6d-4140-920d-1f049c4c6da3 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 201 len: 401 microversion: 2.20 time: 0.047485 Sep 25 01:00:24.174815 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 292/574] 10.222.0.52 () {62 vars in 1200 bytes} [Sun Sep 25 01:00:24 2022] POST /compute/v2.1/os-keypairs => generated 401 bytes in 49 msecs (HTTP/1.1 201) 9 headers in 364 bytes (1 switches on core 0) Sep 25 01:00:25.507778 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeShelveTestJSON-server-1406966367", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "334cdfa6-14bd-4f9b-b93c-dbd866cdb6bb"}], "security_groups": [{"name": "tempest-securitygroup--1992263861"}], "key_name": "tempest-keypair-2072723538", "user_data": "}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:25.620915 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] validate_networks() for [('334cdfa6-14bd-4f9b-b93c-dbd866cdb6bb', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:00:26.230278 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:00:26.230700 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:00:26.232332 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:00:26.232741 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:00:26.233150 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:00:26.242244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:26.242716 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:26.243148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:26.243866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:26.244311 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:26.244727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:26.265901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Getting quotas for project f09655506e354c9d991f4df9a92f4086. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:00:26.269917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Getting quotas for user dbe551f8c10c4111b4ad7380a6482564 and project f09655506e354c9d991f4df9a92f4086. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:00:26.281661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:00:26.283965 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:00:26.337116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-02efd5f4-6bd7-478b-83e4-1e9bed3f3dff tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.20 time: 0.834684 Sep 25 01:00:26.337728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 283/575] 10.222.0.52 () {62 vars in 1188 bytes} [Sun Sep 25 01:00:25 2022] POST /compute/v2.1/servers => generated 402 bytes in 836 msecs (HTTP/1.1 202) 10 headers in 453 bytes (1 switches on core 0) Sep 25 01:00:26.349751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1d01f9e0-52e3-42a5-bd00-75a72172e63f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:26.350403 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1d01f9e0-52e3-42a5-bd00-75a72172e63f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:26.352418 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=95,nova_api:INSERT=4,nova_api:UPDATE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:26.413175 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1d01f9e0-52e3-42a5-bd00-75a72172e63f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:26.422024 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1d01f9e0-52e3-42a5-bd00-75a72172e63f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:26.423465 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1d01f9e0-52e3-42a5-bd00-75a72172e63f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:26.424618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1d01f9e0-52e3-42a5-bd00-75a72172e63f tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1282 microversion: 2.20 time: 0.079171 Sep 25 01:00:26.425103 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 293/576] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:26 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1282 bytes in 80 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:27.450782 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:27.451431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:27.459319 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:27.459886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:27.460393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:27.461147 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:27.461685 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:27.462164 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:27.536939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:27.546019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:27.547468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:27.548593 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-414d70ab-5317-4380-ae13-54b92aac0e77 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1334 microversion: 2.20 time: 0.102083 Sep 25 01:00:27.549148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 284/577] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:27 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1334 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:28.580991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:28.581863 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:28.589246 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:28.589869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:28.591539 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:28.592844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:28.593521 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:28.594149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:28.691771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:28.709312 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:28.712946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:28.714429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0294bdb7-93f3-4c6b-b7b5-dadbfff1addb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1340 microversion: 2.20 time: 0.140511 Sep 25 01:00:28.715550 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 294/578] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:28 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1340 bytes in 143 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:29.697411 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-fdbd1e5d-1e55-490a-9783-7df6b4aad6a1 req-c995e9b7-cbac-456a-9d27-8b9770caf19e service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4", "tag": "4aef1009-d2d3-4c6f-af8a-c5134e19fa37"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:29.713217 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-fdbd1e5d-1e55-490a-9783-7df6b4aad6a1 req-c995e9b7-cbac-456a-9d27-8b9770caf19e service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:29.713910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-fdbd1e5d-1e55-490a-9783-7df6b4aad6a1 req-c995e9b7-cbac-456a-9d27-8b9770caf19e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:29.714433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-fdbd1e5d-1e55-490a-9783-7df6b4aad6a1 req-c995e9b7-cbac-456a-9d27-8b9770caf19e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:29.729083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-fdbd1e5d-1e55-490a-9783-7df6b4aad6a1 req-c995e9b7-cbac-456a-9d27-8b9770caf19e service nova] Creating event network-changed:4aef1009-d2d3-4c6f-af8a-c5134e19fa37 for instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 on hv-cinder-maste Sep 25 01:00:29.737942 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-fdbd1e5d-1e55-490a-9783-7df6b4aad6a1 req-c995e9b7-cbac-456a-9d27-8b9770caf19e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.044932 Sep 25 01:00:29.739158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 285/579] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:29 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:29.760676 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:29.761377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:29.768487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:29.768986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:29.769455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:29.770164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:29.770652 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:29.771076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:30.001096 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:30.010597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:30.012203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:30.013549 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-bb68b123-34da-4ccd-807b-5cdd6215c491 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1408 microversion: 2.20 time: 0.259119 Sep 25 01:00:30.014233 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 295/580] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:29 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1408 bytes in 261 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:31.040672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:31.041571 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:31.050030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:31.050646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:31.051297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:31.052098 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:31.052669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:31.053132 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:31.204565 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:31.213744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:31.215310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:31.216493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9ed45b8f-cf5c-4d57-bb91-c1fb89015107 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1408 microversion: 2.20 time: 0.180729 Sep 25 01:00:31.217142 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 286/581] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:31 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1408 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:32.259177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:32.259893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:32.267542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:32.268041 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:32.268511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:32.269316 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:32.269775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:32.270207 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:32.424119 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:32.432844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:32.434360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:32.435475 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e48ad059-c1d1-4d9b-a836-87082d2a51dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1408 microversion: 2.20 time: 0.180191 Sep 25 01:00:32.436026 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 296/582] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:32 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1408 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:33.461235 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:33.461967 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:33.470558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:33.471084 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:33.471524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:33.472270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:33.472728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:33.473156 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:33.616328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:33.625524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:33.626968 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:33.628091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d94a7b70-870b-4960-ae9d-f9fa7fd76fc6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1408 microversion: 2.20 time: 0.170861 Sep 25 01:00:33.628655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 287/583] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:33 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1408 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:34.653958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:34.654572 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:34.660744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:34.661233 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:34.661699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:34.662404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:34.662855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:34.663284 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:34.824976 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:34.833938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:34.835382 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:34.836509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a2d17476-fb77-4fdf-b881-3e062e83ef44 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1408 microversion: 2.20 time: 0.187067 Sep 25 01:00:34.837078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 297/584] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:34 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1408 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:00:35.863402 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:35.864006 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:35.871491 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:35.871901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:35.872862 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:35.873024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:35.873457 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:35.873897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:36.028087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:36.037890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:36.039352 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:36.040483 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-461fc8cb-b0ca-4341-a11b-248324c1cc49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1408 microversion: 2.20 time: 0.181467 Sep 25 01:00:36.041047 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 288/585] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:35 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1408 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:00:36.124278 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-9c22421c-5e4c-41c4-a157-86bbaea0b203 req-668eb60a-316e-47b6-b2e2-d11842a363bc service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4", "name": "network-vif-plugged", "status": "completed", "tag": "4aef1009-d2d3-4c6f-af8a-c5134e19fa37"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:36.134724 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-9c22421c-5e4c-41c4-a157-86bbaea0b203 req-668eb60a-316e-47b6-b2e2-d11842a363bc service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:36.135209 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-9c22421c-5e4c-41c4-a157-86bbaea0b203 req-668eb60a-316e-47b6-b2e2-d11842a363bc service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:36.135652 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-9c22421c-5e4c-41c4-a157-86bbaea0b203 req-668eb60a-316e-47b6-b2e2-d11842a363bc service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:36.148018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-9c22421c-5e4c-41c4-a157-86bbaea0b203 req-668eb60a-316e-47b6-b2e2-d11842a363bc service nova] Creating event network-vif-plugged:4aef1009-d2d3-4c6f-af8a-c5134e19fa37 for instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 on hv-cinder-maste Sep 25 01:00:36.153855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-9c22421c-5e4c-41c4-a157-86bbaea0b203 req-668eb60a-316e-47b6-b2e2-d11842a363bc service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.033299 Sep 25 01:00:36.154465 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 298/586] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:36 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 35 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:37.065500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:37.066126 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:37.068101 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:37.068985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=6 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:37.076583 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:37.077159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:37.077617 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:37.078535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:37.078902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:37.079295 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:37.227508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:37.236625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:37.238088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:37.239226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c1cb3b95-0bdc-449c-bf14-bd019d7abd1b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1408 microversion: 2.20 time: 0.177682 Sep 25 01:00:37.239798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 289/587] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:37 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1408 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:00:38.261483 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=6 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:38.267395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:38.268254 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:38.275253 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:38.275773 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:38.276216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:38.276921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:38.277453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:38.278020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:38.453206 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:38.468570 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:38.471243 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:38.473427 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ec74f673-1a86-4010-b689-e6c170562335 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1408 microversion: 2.20 time: 0.212745 Sep 25 01:00:38.474445 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 299/588] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:38 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1408 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:39.519475 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:39.520159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:39.527392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:39.527888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:39.528323 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:39.529034 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:39.530399 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:39.530830 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:39.675700 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:39.684865 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:39.686364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:39.687530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d8c1d9ae-93b7-4215-a2c7-68d3d9511b11 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1600 microversion: 2.20 time: 0.172002 Sep 25 01:00:39.688097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 290/589] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:00:39 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1600 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:00:39.717949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:39.718849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:39.730057 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:39.731511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:39.732315 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:39.733686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:39.734521 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:39.735287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:39.823466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0c87f6f9-e28b-4a6b-98b4-00c8f3e48c70 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/os-interface" status: 200 len: 284 microversion: 2.20 time: 0.114162 Sep 25 01:00:39.824118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 300/590] 10.222.0.52 () {60 vars in 1316 bytes} [Sun Sep 25 01:00:39 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/os-interface => generated 284 bytes in 116 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 01:00:41.193846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-4663a7d2-2843-4b9a-9b3e-25daf154f4f6 req-f70eeb23-03e1-453f-a4ea-36a2d4e24a38 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4", "tag": "4aef1009-d2d3-4c6f-af8a-c5134e19fa37"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:41.204855 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-4663a7d2-2843-4b9a-9b3e-25daf154f4f6 req-f70eeb23-03e1-453f-a4ea-36a2d4e24a38 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:41.205371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-4663a7d2-2843-4b9a-9b3e-25daf154f4f6 req-f70eeb23-03e1-453f-a4ea-36a2d4e24a38 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:41.205812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-4663a7d2-2843-4b9a-9b3e-25daf154f4f6 req-f70eeb23-03e1-453f-a4ea-36a2d4e24a38 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:41.219128 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-4663a7d2-2843-4b9a-9b3e-25daf154f4f6 req-f70eeb23-03e1-453f-a4ea-36a2d4e24a38 service nova] Creating event network-changed:4aef1009-d2d3-4c6f-af8a-c5134e19fa37 for instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 on hv-cinder-maste Sep 25 01:00:41.224775 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-4663a7d2-2843-4b9a-9b3e-25daf154f4f6 req-f70eeb23-03e1-453f-a4ea-36a2d4e24a38 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.034964 Sep 25 01:00:41.225815 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 291/591] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:41 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:51.197115 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=17 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:51.198128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=86,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:51.204097 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:51.204674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:51.213530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:51.214866 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:51.215354 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:51.216089 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:51.216554 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:51.216977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:51.555332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lazy-loading 'services' on Instance uuid 621dfc45-048c-452f-86ca-ffdaeeb5f919 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:00:51.657859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8f3e0e8f-4ecf-4129-ba7f-c96536c83ce7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "DELETE /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments/0ff9470c-98a0-4477-980a-1fb9e8d393c5" status: 202 len: 0 microversion: 2.1 time: 0.460870 Sep 25 01:00:51.658924 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 301/592] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:00:51 2022] DELETE /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments/0ff9470c-98a0-4477-980a-1fb9e8d393c5 => generated 0 bytes in 463 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:00:51.667363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=53,nova_api:DELETE=1,nova_api:INSERT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:51.670901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=48 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:00:51.678960 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:51.684284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:51.697405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:51.698287 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:51.700033 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:51.701376 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:51.702230 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:51.703019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:51.768943 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9a3e3d6d-b77e-4f8d-9547-cbfbe6238f6f tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.103822 Sep 25 01:00:51.770308 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 292/593] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:00:51 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments => generated 197 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:00:52.788138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:52.791397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:52.800640 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:52.801163 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:52.801714 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:52.802488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:52.802946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:52.803370 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:52.838497 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-88633ed2-a648-43f4-81db-797875a57eaf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056336 Sep 25 01:00:52.839127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 302/594] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:00:52 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:53.853013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:53.855529 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:53.863263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:53.863846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:53.864463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:53.865320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:53.865848 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:53.866320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:53.906222 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-34322595-de38-4b54-b985-524621692c55 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.057345 Sep 25 01:00:53.906777 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 293/595] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:00:53 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919/os-volume_attachments => generated 25 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 25 01:00:54.004965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:54.005606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:54.012880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:54.013672 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:54.014257 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:54.015339 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:54.016076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:54.016634 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:54.054088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:00:54.175216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-91038c37-d92f-4109-825d-f60b109a84e1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "DELETE /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 204 len: 0 microversion: 2.1 time: 0.173655 Sep 25 01:00:54.176181 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 303/596] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:00:53 2022] DELETE /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:00:54.195183 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:54.196226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:54.203372 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:54.203864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:54.204303 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:54.205030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:54.205542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:54.205964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:54.508667 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:54.519309 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-018917cb-7ea7-4e74-9a9a-62341dfd94c6 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1691 microversion: 2.1 time: 0.334180 Sep 25 01:00:54.520055 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 294/597] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:54 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1691 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:00:54.611293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:54.612256 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:54.623394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:54.624208 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:54.624978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:54.626272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:54.627086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:54.627801 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:54.660008 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2a4c8f72-ff98-4f3a-85b5-c57a7efaf3b9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/ips" status: 200 len: 150 microversion: 2.1 time: 0.119609 Sep 25 01:00:54.660574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 304/598] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 01:00:54 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/ips => generated 150 bytes in 121 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:55.557393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:55.559623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:55.567249 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:55.567839 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:55.568347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:55.569140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:55.569717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:55.570213 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:55.738119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:55.748317 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b28c3743-5e76-4745-8ebc-3a0eb0d42cdf tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1691 microversion: 2.1 time: 0.195014 Sep 25 01:00:55.748907 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 295/599] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:55 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1691 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:56.103719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2d507124-175a-4f5c-87dd-f316c7bfab6f"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:56.106888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:56.115487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:56.115985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:56.116548 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:56.117301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:56.117762 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:56.118187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:56.841061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:56.841726 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:56.848846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:56.849476 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:56.849974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:56.850703 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:56.851296 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:56.851783 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:57.007988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:57.017682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-75be1c06-37da-4ec5-972c-0e59ddf6fb5a tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1691 microversion: 2.1 time: 0.247752 Sep 25 01:00:57.018271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 296/600] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:56 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1691 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:58.046904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:58.048140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:58.057910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:58.059106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:58.060210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:58.061168 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:58.061625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:58.062369 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:58.071609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fa903af3-9f88-47ea-a857-3f353f7ad0a9 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 1.963667 Sep 25 01:00:58.077531 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 305/601] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 01:00:56 2022] POST /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments => generated 194 bytes in 1980 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:58.309967 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:58.325180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e11811a4-be9c-4bcd-a412-3ae3e379ae44 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1691 microversion: 2.1 time: 0.285145 Sep 25 01:00:58.326208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 297/602] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:58 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1691 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:00:59.375556 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-90dd16ce-0da5-46a7-8505-609dec98cc24 req-2d6295be-591f-4b23-9169-2d0155f064ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "621dfc45-048c-452f-86ca-ffdaeeb5f919", "name": "network-vif-deleted", "tag": "a554103e-40f9-42b8-b04a-4fc7d8888240"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:00:59.389006 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-90dd16ce-0da5-46a7-8505-609dec98cc24 req-2d6295be-591f-4b23-9169-2d0155f064ea service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:59.389982 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-90dd16ce-0da5-46a7-8505-609dec98cc24 req-2d6295be-591f-4b23-9169-2d0155f064ea service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:59.390759 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-90dd16ce-0da5-46a7-8505-609dec98cc24 req-2d6295be-591f-4b23-9169-2d0155f064ea service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:59.398918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:00:59.402440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:00:59.416631 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:59.417614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:59.418262 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:59.419216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:00:59.419893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:00:59.420887 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:00:59.433553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-90dd16ce-0da5-46a7-8505-609dec98cc24 req-2d6295be-591f-4b23-9169-2d0155f064ea service nova] Creating event network-vif-deleted:a554103e-40f9-42b8-b04a-4fc7d8888240 for instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 on hv-cinder-maste Sep 25 01:00:59.446577 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-90dd16ce-0da5-46a7-8505-609dec98cc24 req-2d6295be-591f-4b23-9169-2d0155f064ea service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.073766 Sep 25 01:00:59.447534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 298/603] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:00:59 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:00:59.514173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:00:59.523696 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-64e07b3a-9a57-47ff-ab21-eba8dcfc27a7 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 200 len: 1331 microversion: 2.1 time: 0.158144 Sep 25 01:00:59.524985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 306/604] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:00:59 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 1331 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:00.565089 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:00.565864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 621dfc45-048c-452f-86ca-ffdaeeb5f919] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:00.574698 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:00.575572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:00.576347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:00.577711 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:00.578497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:00.579141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:00.598216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] HTTP exception thrown: Instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 could not be found. Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Request method failure captured: Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 HTTP/1.1 Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cb8QFEWahSITkrpCBqxYYBD9zi_oLSAVWAk9whdwsp48CleshPfYnV97ECBsJa63kh6to7hObTL2AlKDs6Ggck8j1eD8_mIGQLRF0aMsS9KlL5zrlzoseQDGerOSLOpzlzHV2LyCeRa0pLQWwIucZcMEZw7qNiXQLyFloWXCSSLWQ Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 28568584195d475fbc52d76185e09044 Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:01:00.601417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_28568584195d475fbc52d76185e09044", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/28568584195d475fbc52d76185e09044"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}] Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 28568584195d475fbc52d76185e09044 Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 8eabe2a74a604c47bc2e5c88b960dcb6 Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 could not be found. Sep 25 01:01:00.605769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '621dfc45-048c-452f-86ca-ffdaeeb5f919'} Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 could not be found. Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:01:00.608064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 could not be found. Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 could not be found. Sep 25 01:01:00.611298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:00.614410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Returning 404 to user: Instance 621dfc45-048c-452f-86ca-ffdaeeb5f919 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:01:00.614410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-209f1364-73d4-42e9-ae0d-5fe077759f26 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919" status: 404 len: 111 microversion: 2.1 time: 0.043933 Sep 25 01:01:00.614410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 299/605] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:00 2022] GET /compute/v2.1/servers/621dfc45-048c-452f-86ca-ffdaeeb5f919 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:01:00.616773 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6fee7b20-832b-4866-b487-8d43973c1585 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '.version_select at 0x7fe22db65310>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:00.648076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6fee7b20-832b-4866-b487-8d43973c1585 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1456756188" status: 202 len: 0 microversion: 2.1 time: 0.036983 Sep 25 01:01:00.648833 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 307/606] 10.222.0.52 () {58 vars in 1221 bytes} [Sun Sep 25 01:01:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1456756188 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 01:01:03.877617 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:03.878484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:03.887906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:03.888518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:03.889006 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:03.889816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:03.890317 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:03.890799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:03.923072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Action: 'action', calling method: >, body: {"removeSecurityGroup": {"name": "tempest-secgroup-smoke-211599788"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:03.923877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:01:03.924237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:03.937978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:03.938789 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:03.939548 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:03.940817 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:03.941662 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:03.942452 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:04.043841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-13511131-2380-4e11-b43d-f438674e666e tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "DELETE /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 204 len: 0 microversion: 2.1 time: 0.170272 Sep 25 01:01:04.044737 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 300/607] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:01:03 2022] DELETE /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:01:04.056030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:04.057222 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:04.068892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:04.069804 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:04.070592 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:04.071600 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:04.072080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:04.072517 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:04.156325 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.network.security_group_api [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Removing security group 42581346-2174-404e-9723-a7e42899addf from port 0940e2bc-bbba-4cec-82c5-38b72d995723 Sep 25 01:01:04.267495 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:04.282301 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c68dc29f-1c4d-492b-ab5e-c356c42fb365 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1507 microversion: 2.1 time: 0.231994 Sep 25 01:01:04.283418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 301/608] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:04 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1507 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:04.586273 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f65b663f-ca94-4205-9f79-1216f5b41d77 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "POST /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/action" status: 202 len: 0 microversion: 2.1 time: 0.670752 Sep 25 01:01:04.587275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 308/609] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:01:03 2022] POST /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/action => generated 0 bytes in 673 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 01:01:04.601799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-12b9b52c-46ca-4a1a-8583-4ecc155d5047 req-ada46d1a-d450-4a99-8a94-09f97267c0cb service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27675569-1b42-49db-a883-3ad6e55fc75d", "tag": "0940e2bc-bbba-4cec-82c5-38b72d995723"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:04.626847 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-12b9b52c-46ca-4a1a-8583-4ecc155d5047 req-ada46d1a-d450-4a99-8a94-09f97267c0cb service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:04.627706 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-12b9b52c-46ca-4a1a-8583-4ecc155d5047 req-ada46d1a-d450-4a99-8a94-09f97267c0cb service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:04.628460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-12b9b52c-46ca-4a1a-8583-4ecc155d5047 req-ada46d1a-d450-4a99-8a94-09f97267c0cb service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:04.651909 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-12b9b52c-46ca-4a1a-8583-4ecc155d5047 req-ada46d1a-d450-4a99-8a94-09f97267c0cb service nova] Creating event network-changed:0940e2bc-bbba-4cec-82c5-38b72d995723 for instance 27675569-1b42-49db-a883-3ad6e55fc75d on hv-cinder-maste Sep 25 01:01:04.661668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-12b9b52c-46ca-4a1a-8583-4ecc155d5047 req-ada46d1a-d450-4a99-8a94-09f97267c0cb service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065202 Sep 25 01:01:04.663261 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 302/610] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:04 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:05.308647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:05.309605 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:05.317374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:05.318025 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:05.318601 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:05.319464 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:05.320036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:05.320593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:05.600539 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:05.616140 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4068c4fb-5fb9-46dd-958d-a32d022d005b tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1507 microversion: 2.1 time: 0.312462 Sep 25 01:01:05.617218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 309/611] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:05 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1507 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:06.141247 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:06.142018 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:06.148644 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:06.149133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:06.149623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:06.150347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:06.150840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:06.151532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:06.249426 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lazy-loading 'services' on Instance uuid 27675569-1b42-49db-a883-3ad6e55fc75d {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:01:06.337071 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a2fc019e-eef9-4633-9de5-fc3e4dd2bbd1 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "DELETE /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments/a2c4fd58-bcf5-4e8b-ac9d-746e34799641" status: 202 len: 0 microversion: 2.1 time: 0.201494 Sep 25 01:01:06.337697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 303/612] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:01:06 2022] DELETE /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments/a2c4fd58-bcf5-4e8b-ac9d-746e34799641 => generated 0 bytes in 204 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:01:06.351088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:06.355431 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:06.367110 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:06.368063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:06.368929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:06.370921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:06.371872 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:06.372711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:06.411388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-53fa979f-0f47-453c-8fe8-3723896ce8a3 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.067768 Sep 25 01:01:06.411987 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 310/613] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:01:06 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments => generated 197 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:06.655876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:06.656606 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:06.664659 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:06.665469 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:06.666139 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:06.667673 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:06.668253 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:06.668827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:06.712276 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-414d829d-a459-40f3-8b12-3deddeca14ef req-b78835b7-176d-4cf1-8521-4a59d5d77839 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "27675569-1b42-49db-a883-3ad6e55fc75d", "tag": "0940e2bc-bbba-4cec-82c5-38b72d995723"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:06.731356 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-414d829d-a459-40f3-8b12-3deddeca14ef req-b78835b7-176d-4cf1-8521-4a59d5d77839 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:06.732296 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-414d829d-a459-40f3-8b12-3deddeca14ef req-b78835b7-176d-4cf1-8521-4a59d5d77839 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:06.733145 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-414d829d-a459-40f3-8b12-3deddeca14ef req-b78835b7-176d-4cf1-8521-4a59d5d77839 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:06.758243 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-414d829d-a459-40f3-8b12-3deddeca14ef req-b78835b7-176d-4cf1-8521-4a59d5d77839 service nova] Creating event network-changed:0940e2bc-bbba-4cec-82c5-38b72d995723 for instance 27675569-1b42-49db-a883-3ad6e55fc75d on hv-cinder-maste Sep 25 01:01:06.768148 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-414d829d-a459-40f3-8b12-3deddeca14ef req-b78835b7-176d-4cf1-8521-4a59d5d77839 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.065364 Sep 25 01:01:06.769479 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 311/614] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:06 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:06.868558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:06.879379 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9405b712-3353-4ead-be4b-308c5f8d41d2 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1507 microversion: 2.1 time: 0.228292 Sep 25 01:01:06.893741 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 304/615] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:06 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1507 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:07.430519 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:07.434698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:07.446282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:07.447169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:07.448769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:07.450156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:07.450984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:07.451885 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:07.557312 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-250ffea9-1c25-481e-8344-61b67e74ef8e tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.135971 Sep 25 01:01:07.558880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 312/616] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:01:07 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments => generated 197 bytes in 139 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:07.912071 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:07.913564 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:07.926684 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:07.927589 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:07.929698 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:07.931652 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:07.933015 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:07.934094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:08.076778 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:08.100479 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2dd7df3d-5d4d-4749-b4b3-14593a10e2fc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1465 microversion: 2.1 time: 0.193657 Sep 25 01:01:08.106882 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 305/617] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:07 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1465 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:08.183630 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-06e3790c-37c8-4fea-af8b-b95731c8c02f req-25b4f9a5-02f3-4932-9638-9a4218c5ab5b service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "67d7a7f9-a0d9-4f96-b9b2-569049374b4f", "name": "network-vif-deleted", "tag": "344eed51-ca03-4da0-b7a1-415ff8227472"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:08.204351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-06e3790c-37c8-4fea-af8b-b95731c8c02f req-25b4f9a5-02f3-4932-9638-9a4218c5ab5b service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:08.204995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-06e3790c-37c8-4fea-af8b-b95731c8c02f req-25b4f9a5-02f3-4932-9638-9a4218c5ab5b service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:08.205631 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-06e3790c-37c8-4fea-af8b-b95731c8c02f req-25b4f9a5-02f3-4932-9638-9a4218c5ab5b service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:08.232279 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-06e3790c-37c8-4fea-af8b-b95731c8c02f req-25b4f9a5-02f3-4932-9638-9a4218c5ab5b service nova] Creating event network-vif-deleted:344eed51-ca03-4da0-b7a1-415ff8227472 for instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f on hv-cinder-maste Sep 25 01:01:08.240095 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-06e3790c-37c8-4fea-af8b-b95731c8c02f req-25b4f9a5-02f3-4932-9638-9a4218c5ab5b service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.061294 Sep 25 01:01:08.241360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 313/618] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:08 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:01:08.265251 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2d507124-175a-4f5c-87dd-f316c7bfab6f"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:08.271031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:08.284054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:08.285246 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:08.286330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:08.287907 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:08.289971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:08.291095 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:08.505152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Invalid volume: volume 2d507124-175a-4f5c-87dd-f316c7bfab6f is already attached to instances: e53662f0-18c6-40e1-99da-07e0065395e1 Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: POST /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments HTTP/1.1 Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 74 Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:01:08.507511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {"volumeAttachment": {"volumeId": "2d507124-175a-4f5c-87dd-f316c7bfab6f"}} Sep 25 01:01:08.509961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Invalid volume: volume 2d507124-175a-4f5c-87dd-f316c7bfab6f is already attached to instances: e53662f0-18c6-40e1-99da-07e0065395e1 Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'server_id': 'e53662f0-18c6-40e1-99da-07e0065395e1', 'body': {'volumeAttachment': {'volumeId': '2d507124-175a-4f5c-87dd-f316c7bfab6f'}}} Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 388, in create Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi device = self.compute_api.attach_volume( Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 241, in inner Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return function(self, context, instance, *args, **kwargs) Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 167, in inner Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(self, context, instance, *args, **kw) Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 5130, in attach_volume Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._check_volume_already_attached(context, instance, volume) Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 4970, in _check_volume_already_attached Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InvalidVolume(reason=msg) Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InvalidVolume: Invalid volume: volume 2d507124-175a-4f5c-87dd-f316c7bfab6f is already attached to instances: e53662f0-18c6-40e1-99da-07e0065395e1 Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:08.512135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 110, in wrapper Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func(*args, **kwargs) Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 406, in create Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPBadRequest(explanation=e.format_message()) Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPBadRequest: Invalid volume: volume 2d507124-175a-4f5c-87dd-f316c7bfab6f is already attached to instances: e53662f0-18c6-40e1-99da-07e0065395e1 Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:08.516995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:08.523131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:08.523131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:01:08.523131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:01:08.523131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid volume: volume 2d507124-175a-4f5c-87dd-f316c7bfab6f is already attached to instances: e53662f0-18c6-40e1-99da-07e0065395e1 Sep 25 01:01:08.523131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:08.523131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 400 to user: Invalid volume: volume 2d507124-175a-4f5c-87dd-f316c7bfab6f is already attached to instances: e53662f0-18c6-40e1-99da-07e0065395e1 {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:01:08.523131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-72300291-375f-4108-ad7e-f0323e5ea969 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments" status: 400 len: 174 microversion: 2.1 time: 0.254268 Sep 25 01:01:08.523131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 306/619] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 01:01:08 2022] POST /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments => generated 174 bytes in 256 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 25 01:01:08.576830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:08.580906 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:08.612626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:08.614180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:08.614387 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:08.615796 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:08.616198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:08.617603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:08.618684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:08.620173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:08.639394 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:08.640753 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:08.647210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:08.648719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:08.649951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:08.651343 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:08.688820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1831ee59-8da4-433e-b10b-40066446b17a tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.120244 Sep 25 01:01:08.692783 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 314/620] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:01:08 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments => generated 197 bytes in 125 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:08.845444 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lazy-loading 'services' on Instance uuid e53662f0-18c6-40e1-99da-07e0065395e1 {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:01:08.871439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=1,nova_cell1:INSERT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:08.978076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4fb4e805-94c0-4562-9202-00e4b57fabb1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments/2d507124-175a-4f5c-87dd-f316c7bfab6f" status: 202 len: 0 microversion: 2.1 time: 0.371594 Sep 25 01:01:08.980457 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 307/621] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:01:08 2022] DELETE /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments/2d507124-175a-4f5c-87dd-f316c7bfab6f => generated 0 bytes in 375 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:01:08.988632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:08.991160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:08.998168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:08.998770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:08.999322 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.000344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:09.000886 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:09.001429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.040378 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-61e55eb5-0e30-49d7-9219-31cac83244b4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.056179 Sep 25 01:01:09.041014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 315/622] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:01:08 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments => generated 197 bytes in 58 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:09.153543 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:09.154412 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:09.161618 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:09.162190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:09.162785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.163573 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:09.164218 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:09.164799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.204297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:09.204788 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:09.205239 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.205735 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lazy-loading 'fault' on Instance uuid 67d7a7f9-a0d9-4f96-b9b2-569049374b4f {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:01:09.287290 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:09.298531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-06b21403-baf5-43eb-9b3c-52cb4dd4fbb1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 200 len: 1311 microversion: 2.1 time: 0.149387 Sep 25 01:01:09.299263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 308/623] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:09 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 1311 bytes in 151 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:09.707492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:09.710194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:09.718784 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:09.719421 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:09.719901 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.720655 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:09.721162 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:09.721658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.750895 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=94,nova_cell1:INSERT=3,nova_cell1:UPDATE=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:09.756913 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4466b195-5dec-4a8e-a1ec-6d7594562afc tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.053518 Sep 25 01:01:09.757656 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 316/624] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:01:09 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments => generated 25 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 25 01:01:09.774629 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:09.775665 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:09.785717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:09.786694 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:09.787599 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.789010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:09.789928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:09.790560 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:09.894404 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] HTTP exception thrown: Instance 27675569-1b42-49db-a883-3ad6e55fc75d is not attached to volume a2c4fd58-bcf5-4e8b-ac9d-746e34799641 Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Request method failure captured: Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments/a2c4fd58-bcf5-4e8b-ac9d-746e34799641 HTTP/1.1 Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6ee6iOqbFhtwP6-ja9YhHEILvC2m5-UMBNSbDM7qktuoBzsGJgK-A-e_b0kgU7MB5KeErRrxJYRZDjBu7qQbnL-qmC0QeV_ErNkcUT_mRenDdvjZT6csbigwBgtMJ_tbMUWNgRAxQkXeOrvVmOzVSjzFWAWPopgRx0Uxg10HSLggic Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: b161c2e2dd1f42159dc0d71e8216516c Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-TestMinimumBasicScenario-178748133 Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:01:09.896958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_b161c2e2dd1f42159dc0d71e8216516c", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/b161c2e2dd1f42159dc0d71e8216516c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/b161c2e2dd1f42159dc0d71e8216516c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/b161c2e2dd1f42159dc0d71e8216516c"}]}] Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-TestMinimumBasicScenario-178748133 Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: b161c2e2dd1f42159dc0d71e8216516c Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-TestMinimumBasicScenario-178748133 Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-TestMinimumBasicScenario-178748133-project Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 80dd52f1455f44b587716a41e632a155 Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-TestMinimumBasicScenario-178748133-project Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:01:09.900330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 27675569-1b42-49db-a883-3ad6e55fc75d is not attached to volume a2c4fd58-bcf5-4e8b-ac9d-746e34799641 Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'server_id': '27675569-1b42-49db-a883-3ad6e55fc75d', 'id': 'a2c4fd58-bcf5-4e8b-ac9d-746e34799641'} Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 552, in delete Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi bdm = objects.BlockDeviceMapping.get_by_volume_and_instance( Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/oslo_versionedobjects/base.py", line 184, in wrapper Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi result = fn(cls, context, *args, **kwargs) Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/objects/block_device.py", line 293, in get_by_volume_and_instance Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.VolumeBDMNotFound(volume_id=volume_id) Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.VolumeBDMNotFound: No volume Block Device Mapping with id a2c4fd58-bcf5-4e8b-ac9d-746e34799641. Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:01:09.901963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 558, in delete Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=msg) Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27675569-1b42-49db-a883-3ad6e55fc75d is not attached to volume a2c4fd58-bcf5-4e8b-ac9d-746e34799641 Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27675569-1b42-49db-a883-3ad6e55fc75d is not attached to volume a2c4fd58-bcf5-4e8b-ac9d-746e34799641 Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:09.904419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Returning 404 to user: Instance 27675569-1b42-49db-a883-3ad6e55fc75d is not attached to volume a2c4fd58-bcf5-4e8b-ac9d-746e34799641 {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:01:09.907765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9a2ef2b6-c08a-4de2-a047-5ed9a2465402 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "DELETE /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments/a2c4fd58-bcf5-4e8b-ac9d-746e34799641" status: 404 len: 154 microversion: 2.1 time: 0.131613 Sep 25 01:01:09.907765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 309/625] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:01:09 2022] DELETE /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d/os-volume_attachments/a2c4fd58-bcf5-4e8b-ac9d-746e34799641 => generated 154 bytes in 133 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:01:10.063847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:10.068272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:10.080123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:10.080440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:10.080933 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:10.081980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:10.082450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:10.082877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:10.123980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cf2ccb80-860b-4a39-886a-93d7065fbd3b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.069887 Sep 25 01:01:10.124690 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 317/626] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:01:10 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments => generated 197 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:10.344277 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:10.347046 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 67d7a7f9-a0d9-4f96-b9b2-569049374b4f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:10.363274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:10.364136 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:10.364924 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:10.366314 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:10.367173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:10.367997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:10.403658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] HTTP exception thrown: Instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f could not be found. Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Request method failure captured: Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f HTTP/1.1 Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cb8QFEWahSITkrpCBqxYYBD9zi_oLSAVWAk9whdwsp48CleshPfYnV97ECBsJa63kh6to7hObTL2AlKDs6Ggck8j1eD8_mIGQLRF0aMsS9KlL5zrlzoseQDGerOSLOpzlzHV2LyCeRa0pLQWwIucZcMEZw7qNiXQLyFloWXCSSLWQ Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 28568584195d475fbc52d76185e09044 Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:01:10.406648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_28568584195d475fbc52d76185e09044", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/28568584195d475fbc52d76185e09044"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}] Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 28568584195d475fbc52d76185e09044 Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 8eabe2a74a604c47bc2e5c88b960dcb6 Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f could not be found. Sep 25 01:01:10.408170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '67d7a7f9-a0d9-4f96-b9b2-569049374b4f'} Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f could not be found. Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:01:10.409087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f could not be found. Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f could not be found. Sep 25 01:01:10.410487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:10.412255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Returning 404 to user: Instance 67d7a7f9-a0d9-4f96-b9b2-569049374b4f could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:01:10.412255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-138ebe78-bc51-4e65-a7f7-d0175cf9c39d tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f" status: 404 len: 111 microversion: 2.1 time: 0.072410 Sep 25 01:01:10.412255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 310/627] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:10 2022] GET /compute/v2.1/servers/67d7a7f9-a0d9-4f96-b9b2-569049374b4f => generated 111 bytes in 75 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:01:10.422629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:10.423138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:10.429821 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:10.430337 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:10.430771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:10.431512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:10.432019 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:10.432546 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:10.468098 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:01:10.592708 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-90db4674-d592-458e-8d72-f6a579593ef1 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "DELETE /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 204 len: 0 microversion: 2.1 time: 0.174006 Sep 25 01:01:10.593703 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 318/628] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:01:10 2022] DELETE /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 0 bytes in 178 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:01:10.604436 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:10.605318 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:10.613248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:10.613837 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:10.614278 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:10.615015 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:10.615468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:10.615909 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:10.769805 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:10.779727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2ef5938c-1920-4814-aef5-316b30c11588 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.180156 Sep 25 01:01:10.780313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 311/629] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:10 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:11.124025 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:11.124603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:11.131565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.132189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.132648 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.133241 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.133727 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.134150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.141146 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:11.144236 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:11.151730 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.152612 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.153328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.154083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.154636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.155071 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.167019 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:01:11.192340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e5ff3e70-1302-41db-9592-35395072e667 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.057353 Sep 25 01:01:11.192970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 312/630] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:01:11 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1/os-volume_attachments => generated 25 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 25 01:01:11.264464 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7f74c74b-64cc-43b2-a4df-e9ea9e2fd314 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "DELETE /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 204 len: 0 microversion: 2.1 time: 0.144684 Sep 25 01:01:11.265420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 319/631] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:01:11 2022] DELETE /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:01:11.278938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:11.280088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:11.292334 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.293154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.293880 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.294881 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:11.295141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.295488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:11.296169 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.296953 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.303149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.303685 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.304124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.304852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.305367 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.305787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.341052 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:01:11.446125 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-585b78fa-42c5-4f01-a3db-0c73c3f7b2aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 204 len: 0 microversion: 2.1 time: 0.156951 Sep 25 01:01:11.446840 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 320/632] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:01:11 2022] DELETE /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 25 01:01:11.459613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:11.460703 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:11.468817 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.469374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.469810 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.470569 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.471034 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.471601 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.498920 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:11.511035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-398510f1-0abf-4817-9515-a2cadb9b6abd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1554 microversion: 2.1 time: 0.240170 Sep 25 01:01:11.512110 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 313/633] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:11 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1554 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:11.627072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:11.636316 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-11a757c1-c480-4beb-891a-4ede625da9cf tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1679 microversion: 2.1 time: 0.181550 Sep 25 01:01:11.636906 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 321/634] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:11 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1679 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:11.804100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:11.804768 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:11.811952 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.812445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.812874 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.813620 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:11.814075 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:11.814499 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:11.972051 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:11.981630 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a1b471af-b286-4725-946e-6049bafd6055 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.181335 Sep 25 01:01:11.982237 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 314/635] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:11 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:12.537245 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:12.537908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:12.545837 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:12.546334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:12.546780 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:12.547504 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:12.547968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:12.548399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:12.664597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:12.665785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:12.677522 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:12.678417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:12.679231 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:12.680443 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:12.681327 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:12.682520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:12.710364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:12.720132 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f2a3f27e-9f40-4eb8-a15b-8325d8f3091c tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1554 microversion: 2.1 time: 0.186852 Sep 25 01:01:12.720722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 322/636] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:12 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1554 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:12.874292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:12.882724 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b2a559f2-6a91-4af2-890c-da26dfd98b57 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1679 microversion: 2.1 time: 0.225522 Sep 25 01:01:12.883323 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 315/637] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:12 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1679 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:13.006169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:13.006828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:13.014389 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:13.014884 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:13.015349 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:13.016037 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:13.016496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:13.016925 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:13.191223 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:13.200648 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-18377a90-326e-4e9c-aee8-cd8dd95f702c tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.198287 Sep 25 01:01:13.201311 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 323/638] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:13 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:13.759215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:13.760788 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:13.767936 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:13.768432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:13.768870 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:13.769807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:13.770272 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:13.770724 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:13.911827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:13.913124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:13.926302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:13.927176 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:13.927965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:13.929303 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:13.930140 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:13.930945 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:13.954732 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:13.965890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-38d33119-a440-42e6-8ed2-7e0ddecff027 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1554 microversion: 2.1 time: 0.210651 Sep 25 01:01:13.966709 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 316/639] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:13 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1554 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:14.122373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:14.138123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6794307e-0851-461d-80cb-417323878efd tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1679 microversion: 2.1 time: 0.233735 Sep 25 01:01:14.138949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 324/640] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:13 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1679 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:14.239521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:14.240173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:14.246879 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:14.247375 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:14.247823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:14.248540 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:14.248995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:14.249459 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:14.416949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:14.426632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5f41adb1-a56e-4ade-9089-34d44c2b3db5 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.191035 Sep 25 01:01:14.427485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 317/641] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:14 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:14.993107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:14.993962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:15.002175 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:15.002961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:15.003635 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:15.004774 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:15.005601 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:15.006106 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:15.163706 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:15.164935 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:15.173184 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:15.173860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:15.174445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:15.175416 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:15.176042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:15.176738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:15.189642 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:15.199404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-026eeb52-9662-4a56-a920-ebdc539346d7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1554 microversion: 2.1 time: 0.210311 Sep 25 01:01:15.200114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 325/642] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:14 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1554 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:15.336789 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:15.346332 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6e8e18a4-22cb-42e3-9c61-63d76c5892c3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1679 microversion: 2.1 time: 0.187049 Sep 25 01:01:15.346944 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 318/643] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:15 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1679 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:15.450742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:15.451426 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:15.459523 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:15.460143 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:15.460585 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:15.461433 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:15.461899 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:15.462333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:15.627789 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:15.639680 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-63266c9f-56c4-4658-9180-9e27a680a2dc tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.192872 Sep 25 01:01:15.640338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 326/644] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:15 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:16.239260 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:16.240267 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:16.248072 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:16.248571 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:16.249013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:16.249773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:16.250226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:16.250650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:16.373226 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:16.374172 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:16.381144 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:16.381689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:16.382130 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:16.382866 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:16.383334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:16.383759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:16.429550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:16.448127 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-538dcb5a-0c2d-463c-9d31-2223ed404629 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1554 microversion: 2.1 time: 0.212893 Sep 25 01:01:16.448833 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 319/645] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:16 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1554 bytes in 215 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:16.582612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:16.592089 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4540309c-c864-45b0-a9c9-6df25aadc117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1679 microversion: 2.1 time: 0.223476 Sep 25 01:01:16.592599 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 327/646] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:16 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1679 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:16.667917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:16.668578 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:16.675800 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:16.676508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:16.677294 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:16.678191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:16.678845 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:16.679395 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:16.848580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:16.857765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b65e8b93-d0f6-463e-85a7-c08989d5ddf4 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.195625 Sep 25 01:01:16.858594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 320/647] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:16 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:17.476891 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:17.477983 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:17.485037 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:17.485825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:17.486273 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:17.487002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:17.487468 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:17.487903 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:17.519523 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=3,nova_cell1:INSERT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:17.570196 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:17.571082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:17.580456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:17.581153 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:17.581691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:17.582661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:17.583121 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:17.583543 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:17.618072 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:01:17.753785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-27b6cc80-14e8-4ad4-ada2-93a412cf50cf tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "DELETE /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 204 len: 0 microversion: 2.1 time: 0.189229 Sep 25 01:01:17.755979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 321/648] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:01:17 2022] DELETE /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 25 01:01:17.763878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:17.764841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:17.772350 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:17.772992 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:17.773631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:17.774501 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:17.775044 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:17.776243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:17.856509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:17.865604 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-64206b3e-4629-442d-84a9-6d9b69c235c7 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1554 microversion: 2.1 time: 0.393293 Sep 25 01:01:17.866194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 328/649] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:17 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1554 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:17.872803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:17.873464 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:17.881931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:17.882733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:17.883341 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:17.884227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:17.886436 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:17.887118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:17.982423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:17.987512 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=97,nova_cell1:INSERT=2,nova_cell1:UPDATE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:17.996225 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e4948772-bdee-4c74-8e59-87bdb5901d98 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1679 microversion: 2.1 time: 0.238301 Sep 25 01:01:17.996927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 322/650] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:17 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1679 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:18.003891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:18.004514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:18.016431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:18.017031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:18.017519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:18.018319 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:18.018845 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:18.019280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:18.095312 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:18.111516 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f4b108b1-ca0d-459b-b937-e909caf46577 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1576 microversion: 2.1 time: 0.243832 Sep 25 01:01:18.112336 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 329/651] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:17 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1576 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:18.206723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:18.218882 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6b9d213d-6d36-4f2d-82b4-2d98dd164e30 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.220654 Sep 25 01:01:18.219653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 323/652] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:17 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:18.895014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:18.895581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:18.903597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:18.904103 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:18.904543 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:18.905290 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:18.905759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:18.906188 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:19.035125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:19.036329 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:19.049019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:19.049932 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:19.050699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:19.051964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:19.052814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:19.053627 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:19.066225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:19.077162 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-16a87703-9c00-4b10-8b4c-457f470219dd tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1554 microversion: 2.1 time: 0.186201 Sep 25 01:01:19.085161 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 330/653] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:18 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1554 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:19.141967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:19.142965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:19.153120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:19.154752 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:19.155441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:19.157350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:19.158093 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:19.159366 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:19.388252 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:19.419178 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-527e637e-76d2-4969-b419-59cf9cb1291e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1679 microversion: 2.1 time: 0.389985 Sep 25 01:01:19.438226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 324/654] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:19 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1679 bytes in 412 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:19.449521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:19.450708 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:19.460087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:19.460855 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:19.462527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:19.463000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:19.464040 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:19.464827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:19.622368 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:19.636630 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-477e729a-b25d-4103-af1a-b58d8ef5d250 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1576 microversion: 2.1 time: 0.498488 Sep 25 01:01:19.637526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 331/655] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:19 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1576 bytes in 501 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:19.931625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:19.949933 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-04a79a02-51b9-41ef-a9f2-df35781d9735 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1522 microversion: 2.1 time: 0.506685 Sep 25 01:01:19.966732 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 325/656] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:19 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1522 bytes in 528 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:20.105784 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:20.107614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:20.116173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:20.117347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:20.117831 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:20.118756 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:20.119455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:20.120019 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:20.257824 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-3e48b91e-cafc-4049-87f8-1c39be69d2dc req-dc37cc41-a776-45ed-a030-fbdabfb18ea8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "27675569-1b42-49db-a883-3ad6e55fc75d", "name": "network-vif-deleted", "tag": "0940e2bc-bbba-4cec-82c5-38b72d995723"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:20.258672 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:20.270030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-3e48b91e-cafc-4049-87f8-1c39be69d2dc req-dc37cc41-a776-45ed-a030-fbdabfb18ea8 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:20.270530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-3e48b91e-cafc-4049-87f8-1c39be69d2dc req-dc37cc41-a776-45ed-a030-fbdabfb18ea8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:20.271194 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-3e48b91e-cafc-4049-87f8-1c39be69d2dc req-dc37cc41-a776-45ed-a030-fbdabfb18ea8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:20.279911 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-95f95d68-b2a6-4137-85f0-25fbde9e48eb tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1512 microversion: 2.1 time: 0.180104 Sep 25 01:01:20.280520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 332/657] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:20 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1512 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:20.298022 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-3e48b91e-cafc-4049-87f8-1c39be69d2dc req-dc37cc41-a776-45ed-a030-fbdabfb18ea8 service nova] Creating event network-vif-deleted:0940e2bc-bbba-4cec-82c5-38b72d995723 for instance 27675569-1b42-49db-a883-3ad6e55fc75d on hv-cinder-maste Sep 25 01:01:20.314989 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-3e48b91e-cafc-4049-87f8-1c39be69d2dc req-dc37cc41-a776-45ed-a030-fbdabfb18ea8 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.096952 Sep 25 01:01:20.316307 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 326/658] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:20 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:20.460186 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:20.461839 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:20.476249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:20.477810 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:20.478981 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:20.494979 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:20.497095 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:20.498530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:20.634875 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:20.644705 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cf60933d-24b4-44ec-ba08-cd4a1163a0f3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1613 microversion: 2.1 time: 0.193907 Sep 25 01:01:20.645723 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 333/659] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:20 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1613 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:20.685465 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:20.686571 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:20.695926 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:20.696716 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:20.697534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:20.698391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:20.699310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:20.699930 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:20.991444 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:20.992453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:21.002215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.002834 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.004002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.004906 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.005517 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.006020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.054552 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.055989 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.057127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.057700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lazy-loading 'fault' on Instance uuid 3c9017f3-8bdb-4c13-9c50-646241c8f1ba {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:01:21.123438 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '8eabe2a74a604c47bc2e5c88b960dcb6', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '28568584195d475fbc52d76185e09044', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:21.139362 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-08511b3f-9a0f-4ff2-aded-c0389b2f1a90 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 200 len: 1311 microversion: 2.1 time: 0.153443 Sep 25 01:01:21.140955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 334/660] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:20 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 1311 bytes in 155 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:21.322615 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:21.324138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:21.328822 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=92,nova_api:UPDATE=4,nova_api:DELETE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:21.335893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.336679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.337431 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.339239 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.340044 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.340894 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.405738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.407083 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.407759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.408392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lazy-loading 'fault' on Instance uuid 27675569-1b42-49db-a883-3ad6e55fc75d {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:01:21.418756 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:21.456450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-856b49d4-575f-410b-a4b8-0bdec16b0fa0 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1576 microversion: 2.1 time: 0.775429 Sep 25 01:01:21.458664 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 327/661] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:20 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1576 bytes in 779 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:21.513282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '80dd52f1455f44b587716a41e632a155', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'b161c2e2dd1f42159dc0d71e8216516c', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:21.527489 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a6a4b45d-586f-4e6b-9219-a93d6a386aad tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 200 len: 1348 microversion: 2.1 time: 0.210760 Sep 25 01:01:21.528588 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 335/662] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:21 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 1348 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:21.586860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-d13f18e3-93b8-4862-8064-90abfaeec6a6 req-ad9d6a08-2c14-417c-a4cc-4864d16801d0 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "e53662f0-18c6-40e1-99da-07e0065395e1", "name": "network-vif-deleted", "tag": "dc9429b9-a8b5-45cb-94af-f68e5fc7099c"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:21.609665 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-d13f18e3-93b8-4862-8064-90abfaeec6a6 req-ad9d6a08-2c14-417c-a4cc-4864d16801d0 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.610793 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-d13f18e3-93b8-4862-8064-90abfaeec6a6 req-ad9d6a08-2c14-417c-a4cc-4864d16801d0 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.611575 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-d13f18e3-93b8-4862-8064-90abfaeec6a6 req-ad9d6a08-2c14-417c-a4cc-4864d16801d0 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.638959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-d13f18e3-93b8-4862-8064-90abfaeec6a6 req-ad9d6a08-2c14-417c-a4cc-4864d16801d0 service nova] Creating event network-vif-deleted:dc9429b9-a8b5-45cb-94af-f68e5fc7099c for instance e53662f0-18c6-40e1-99da-07e0065395e1 on hv-cinder-maste Sep 25 01:01:21.646658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-d13f18e3-93b8-4862-8064-90abfaeec6a6 req-ad9d6a08-2c14-417c-a4cc-4864d16801d0 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.185802 Sep 25 01:01:21.647485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 328/663] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:21.690383 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:21.695240 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:21.707152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.707711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.708554 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.709625 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:21.710193 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:21.710889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:21.834137 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:21.855114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-48d9e5bd-f2e5-4598-9649-ec7836ec8906 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 200 len: 1325 microversion: 2.1 time: 0.168749 Sep 25 01:01:21.855543 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 336/664] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:21 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 1325 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:22.186455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:22.189066 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] [instance: 3c9017f3-8bdb-4c13-9c50-646241c8f1ba] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:22.199739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=87,nova_api:UPDATE=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:22.202806 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.204030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.205644 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.207687 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.208716 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.209691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.238282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] HTTP exception thrown: Instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba could not be found. Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Request method failure captured: Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba HTTP/1.1 Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cb8QFEWahSITkrpCBqxYYBD9zi_oLSAVWAk9whdwsp48CleshPfYnV97ECBsJa63kh6to7hObTL2AlKDs6Ggck8j1eD8_mIGQLRF0aMsS9KlL5zrlzoseQDGerOSLOpzlzHV2LyCeRa0pLQWwIucZcMEZw7qNiXQLyFloWXCSSLWQ Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 28568584195d475fbc52d76185e09044 Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:01:22.242114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_28568584195d475fbc52d76185e09044", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/28568584195d475fbc52d76185e09044"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/28568584195d475fbc52d76185e09044"}]}] Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 28568584195d475fbc52d76185e09044 Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-ServerRescueNegativeTestJSON-1085047793 Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 8eabe2a74a604c47bc2e5c88b960dcb6 Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-ServerRescueNegativeTestJSON-1085047793-project Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba could not be found. Sep 25 01:01:22.244682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '3c9017f3-8bdb-4c13-9c50-646241c8f1ba'} Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba could not be found. Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:01:22.245836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba could not be found. Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba could not be found. Sep 25 01:01:22.248248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.252242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] Returning 404 to user: Instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:01:22.252242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-08ec3773-00dc-4e40-a9a5-086436817c76 tempest-ServerRescueNegativeTestJSON-1085047793 tempest-ServerRescueNegativeTestJSON-1085047793-project] 10.222.0.52 "GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba" status: 404 len: 111 microversion: 2.1 time: 0.070178 Sep 25 01:01:22.252242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 329/665] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:22 2022] GET /compute/v2.1/servers/3c9017f3-8bdb-4c13-9c50-646241c8f1ba => generated 111 bytes in 73 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:01:22.490658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:22.496520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:22.505827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-8e3dae5b-af37-47dc-8ed2-b8e60379c9a7 req-8f08d659-9463-4973-8c22-65b6d52f5f40 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "3c9017f3-8bdb-4c13-9c50-646241c8f1ba", "name": "network-vif-deleted", "tag": "1f1b4ee1-4f64-49a9-bb20-210b866dfbd5"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:22.527691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.531038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.532148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.534768 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.537819 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.542958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.563063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8e3dae5b-af37-47dc-8ed2-b8e60379c9a7 req-8f08d659-9463-4973-8c22-65b6d52f5f40 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.566200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8e3dae5b-af37-47dc-8ed2-b8e60379c9a7 req-8f08d659-9463-4973-8c22-65b6d52f5f40 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.567607 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8e3dae5b-af37-47dc-8ed2-b8e60379c9a7 req-8f08d659-9463-4973-8c22-65b6d52f5f40 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.635923 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-8e3dae5b-af37-47dc-8ed2-b8e60379c9a7 req-8f08d659-9463-4973-8c22-65b6d52f5f40 service nova] Creating event network-vif-deleted:1f1b4ee1-4f64-49a9-bb20-210b866dfbd5 for instance 3c9017f3-8bdb-4c13-9c50-646241c8f1ba on hv-cinder-maste Sep 25 01:01:22.658750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-8e3dae5b-af37-47dc-8ed2-b8e60379c9a7 req-8f08d659-9463-4973-8c22-65b6d52f5f40 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.322485 Sep 25 01:01:22.660984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 337/666] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:22 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 326 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:22.670353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:22.678464 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] [instance: 27675569-1b42-49db-a883-3ad6e55fc75d] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:22.694168 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.694990 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.695695 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.696695 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.700255 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.701054 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.735684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] HTTP exception thrown: Instance 27675569-1b42-49db-a883-3ad6e55fc75d could not be found. Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Request method failure captured: Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d HTTP/1.1 Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6ee6iOqbFhtwP6-ja9YhHEILvC2m5-UMBNSbDM7qktuoBzsGJgK-A-e_b0kgU7MB5KeErRrxJYRZDjBu7qQbnL-qmC0QeV_ErNkcUT_mRenDdvjZT6csbigwBgtMJ_tbMUWNgRAxQkXeOrvVmOzVSjzFWAWPopgRx0Uxg10HSLggic Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: b161c2e2dd1f42159dc0d71e8216516c Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-TestMinimumBasicScenario-178748133 Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:01:22.739361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_b161c2e2dd1f42159dc0d71e8216516c", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/b161c2e2dd1f42159dc0d71e8216516c"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/b161c2e2dd1f42159dc0d71e8216516c"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/b161c2e2dd1f42159dc0d71e8216516c"}]}] Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-TestMinimumBasicScenario-178748133 Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: b161c2e2dd1f42159dc0d71e8216516c Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-TestMinimumBasicScenario-178748133 Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-TestMinimumBasicScenario-178748133-project Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 80dd52f1455f44b587716a41e632a155 Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-TestMinimumBasicScenario-178748133-project Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 27675569-1b42-49db-a883-3ad6e55fc75d could not be found. Sep 25 01:01:22.742007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '27675569-1b42-49db-a883-3ad6e55fc75d'} Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 27675569-1b42-49db-a883-3ad6e55fc75d could not be found. Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:01:22.743023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 27675569-1b42-49db-a883-3ad6e55fc75d could not be found. Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 27675569-1b42-49db-a883-3ad6e55fc75d could not be found. Sep 25 01:01:22.745251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.750079 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Returning 404 to user: Instance 27675569-1b42-49db-a883-3ad6e55fc75d could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:01:22.757090 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-64100743-9e2a-4d35-9d43-16b59f2ed090 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d" status: 404 len: 111 microversion: 2.1 time: 0.088351 Sep 25 01:01:22.760123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 338/667] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:22 2022] GET /compute/v2.1/servers/27675569-1b42-49db-a883-3ad6e55fc75d => generated 111 bytes in 98 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:01:22.782124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c469b6e4-0346-4344-acb1-abf68adb8d15 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] Calling method '.version_select at 0x7fe22db653a0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:22.826792 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c469b6e4-0346-4344-acb1-abf68adb8d15 tempest-TestMinimumBasicScenario-178748133 tempest-TestMinimumBasicScenario-178748133-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-600490355" status: 202 len: 0 microversion: 2.1 time: 0.055189 Sep 25 01:01:22.834055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 339/668] 10.222.0.52 () {58 vars in 1269 bytes} [Sun Sep 25 01:01:22 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestMinimumBasicScenario-600490355 => generated 0 bytes in 67 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 01:01:22.894438 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:22.898222 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:22.900858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:22.919852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c0e9a2d4-dbe4-4437-b770-8a36fa530c89 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1224 microversion: 2.1 time: 0.438144 Sep 25 01:01:22.920470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 330/669] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:22 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1224 bytes in 440 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:22.951183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.952967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.957699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.959302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:22.960064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:22.961909 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:22.992348 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 HTTP/1.1 Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:01:22.995742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:01:22.997673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': 'e53662f0-18c6-40e1-99da-07e0065395e1'} Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:01:22.998929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:01:23.001577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:01:23.005716 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:01:23.007203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f94340be-c57a-46f6-b109-6192d96a65eb tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 404 len: 111 microversion: 2.1 time: 0.126413 Sep 25 01:01:23.008763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 340/670] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:22 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 111 bytes in 130 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:01:23.026552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-798a8b06-a493-485e-8153-ce76326eb117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '.version_select at 0x7fe22d9f8e50>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:23.076887 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-798a8b06-a493-485e-8153-ce76326eb117 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-127083720" status: 202 len: 0 microversion: 2.1 time: 0.059307 Sep 25 01:01:23.083553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 331/671] 10.222.0.52 () {58 vars in 1218 bytes} [Sun Sep 25 01:01:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-127083720 => generated 0 bytes in 69 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:01:23.700117 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-4493e62a-d10d-4861-b9c3-41e62db84658 req-a820c5cb-28ad-48f0-814d-2b4993fb6604 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "63e5af09-dbe0-471f-a5bd-9519f11c8d70", "name": "network-vif-deleted", "tag": "2d8c367e-4bc1-45e8-8730-7a0f3f231689"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:23.724771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-4493e62a-d10d-4861-b9c3-41e62db84658 req-a820c5cb-28ad-48f0-814d-2b4993fb6604 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:23.725316 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-4493e62a-d10d-4861-b9c3-41e62db84658 req-a820c5cb-28ad-48f0-814d-2b4993fb6604 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:23.730801 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-4493e62a-d10d-4861-b9c3-41e62db84658 req-a820c5cb-28ad-48f0-814d-2b4993fb6604 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:23.755152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-4493e62a-d10d-4861-b9c3-41e62db84658 req-a820c5cb-28ad-48f0-814d-2b4993fb6604 service nova] Creating event network-vif-deleted:2d8c367e-4bc1-45e8-8730-7a0f3f231689 for instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 on hv-cinder-maste Sep 25 01:01:23.761544 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-4493e62a-d10d-4861-b9c3-41e62db84658 req-a820c5cb-28ad-48f0-814d-2b4993fb6604 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.065612 Sep 25 01:01:23.763374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 341/672] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:23 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:23.976135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:23.979019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:24.005475 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:24.024441 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.018s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:24.028325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:24.029996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:24.039154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:24.041219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:24.143428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:24.145895 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:24.147997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:24.151139 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lazy-loading 'fault' on Instance uuid 63e5af09-dbe0-471f-a5bd-9519f11c8d70 {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:01:24.252199 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:24.262158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3252d7a2-190c-4a21-abbe-da55b3decf36 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 200 len: 1183 microversion: 2.1 time: 0.305177 Sep 25 01:01:24.263213 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 332/673] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:23 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 1183 bytes in 308 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:24.808540 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-dc4667c2-18cd-465d-b2d4-404c3a616327 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: .version_select at 0x7fe22d816d30>, body: {"keypair": {"name": "tempest-keypair-412498492", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBLSiq9TBo5cjrcsN2E2bK4v96igZqMpoKc5p+HppoBeYXefKN61HxiJC3izZ2DMbmptR+hIo2gjyxps6GDWlzyIwRJbhiWW1noOW6G+CUPpI+RlSSCdQ/rgqf8Zc/bbHAQ=="}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:24.833577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-dc4667c2-18cd-465d-b2d4-404c3a616327 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:01:24.841419 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-dc4667c2-18cd-465d-b2d4-404c3a616327 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for user 96386a16f0b54b7c94d32bd948caf118 and project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'key_pairs'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:01:24.878612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-dc4667c2-18cd-465d-b2d4-404c3a616327 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 385 microversion: 2.1 time: 0.075367 Sep 25 01:01:24.880301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 342/674] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 01:01:24 2022] POST /compute/v2.1/os-keypairs => generated 385 bytes in 78 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:01:25.309079 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:25.310291 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:25.326357 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:25.327010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:25.327682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:25.329196 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:25.331839 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:25.334816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:25.355339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] HTTP exception thrown: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Request method failure captured: Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 HTTP/1.1 Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6d9c0dB9tCHvIml0BBsUSlOVo-qLb3eukyQgMLgvyRXH1CyMjUyaQc1f1JKazgmAE4J3bJO5T5Z_kKMpRncWkpX-mTfm_aKx9Cap4bu1LUkSDrvuHpNS_NhBfp3ENvyhM7JnJSpK88-o5SQC6xM3absHOF-r0IffzDxhAp_0Swmsps Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:01:25.357408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_46f19fd9ad1e4880961d1eee75883687", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}] Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: c19b10ebc31244dbb2b1539d94ac281d Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:01:25.358779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '63e5af09-dbe0-471f-a5bd-9519f11c8d70'} Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:01:25.359533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:01:25.361017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:01:25.363327 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Returning 404 to user: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:01:25.364404 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f1d8274c-3f71-492a-a1ad-1af5d5df9db3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 404 len: 111 microversion: 2.1 time: 0.063858 Sep 25 01:01:25.365368 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 333/675] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:25 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:01:25.447440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:25.448996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:25.484083 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:25.486376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:25.486872 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:25.487632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:25.488321 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:25.488758 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:25.520686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-dc58da7e-35d1-4570-9d5b-63d3389c448e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/ips" status: 200 len: 151 microversion: 2.20 time: 0.200377 Sep 25 01:01:25.529457 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 343/676] 10.222.0.52 () {60 vars in 1289 bytes} [Sun Sep 25 01:01:25 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/ips => generated 151 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 01:01:25.669409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-410923799", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1527375203", "security_groups": [{"name": "tempest-secgroup-smoke-1834758444"}], "block_device_mapping_v2": [{"uuid": "b08083ef-2e11-4ed7-87d3-6520c8d3cf21", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "31fa7821-6345-4f48-b070-9ceab6d12adf"}]}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:25.946509 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] validate_networks() for [('31fa7821-6345-4f48-b070-9ceab6d12adf', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:01:27.598825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:01:27.601035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:01:27.609597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:01:27.610639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:01:27.611575 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:01:27.628732 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:27.629220 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:27.629702 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:27.630558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:27.631653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:27.632427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:27.646853 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:27.647566 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:27.660202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:27.660718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:27.661156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:27.662218 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Getting quotas for project 46f19fd9ad1e4880961d1eee75883687. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:01:27.663207 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:27.663684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:27.666851 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:27.673631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Getting quotas for user c19b10ebc31244dbb2b1539d94ac281d and project 46f19fd9ad1e4880961d1eee75883687. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:01:27.775747 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:01:27.778219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='b08083ef-2e11-4ed7-87d3-6520c8d3cf21',volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:01:27.780956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lazy-loading 'flavor' on Instance uuid c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:01:28.199660 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8235015f-9143-44eb-9891-a72fa45bcaa1 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 2.536123 Sep 25 01:01:28.213807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 334/677] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:01:25 2022] POST /compute/v2.1/servers => generated 406 bytes in 2551 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 01:01:28.218924 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-20896737-fbe7-4980-aff4-d241f82d4a41 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:28.219607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-20896737-fbe7-4980-aff4-d241f82d4a41 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:28.377888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-20896737-fbe7-4980-aff4-d241f82d4a41 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:28.390607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-20896737-fbe7-4980-aff4-d241f82d4a41 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1094 microversion: 2.1 time: 0.175387 Sep 25 01:01:28.391838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 335/678] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:28 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1094 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:28.441999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-04b250d9-9ffe-4860-9124-e76caa395be6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/action" status: 202 len: 0 microversion: 2.20 time: 0.799798 Sep 25 01:01:28.449466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 344/679] 10.222.0.52 () {62 vars in 1319 bytes} [Sun Sep 25 01:01:27 2022] POST /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/action => generated 0 bytes in 808 msecs (HTTP/1.1 202) 9 headers in 363 bytes (2 switches on core 0) Sep 25 01:01:28.453675 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:28.454307 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:28.473849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:28.474405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:28.495594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.020s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:28.506242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:28.506242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:28.507205 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:29.017023 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:29.036928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:29.039086 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:29.040612 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e60fef03-f279-4db2-b827-d697cc0f09e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.590830 Sep 25 01:01:29.043559 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 336/680] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:28 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:29.431460 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:29.432437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:29.443859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:29.444559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:29.445383 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:29.446273 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:29.447078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:29.447752 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:29.626845 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:29.664144 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0bfba5af-d1d7-4f04-99f7-82e32ea3905c tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1144 microversion: 2.1 time: 0.236476 Sep 25 01:01:29.680935 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 345/681] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:29 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1144 bytes in 257 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:30.076857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:30.077959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:30.080902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeNegativeTest-server-1278140790", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "5dc49b54-686c-4619-ad4c-65c72c26fd31"}], "security_groups": [{"name": "tempest-securitygroup--399920501"}], "key_name": "tempest-keypair-412498492", "user_data": "}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:30.105984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:30.110667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:30.112669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:30.114284 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:30.115823 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:30.116635 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:30.488089 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:30.504801 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:30.506722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:30.508194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c53b6fa2-0655-42b6-8ecb-caa221a83800 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.435520 Sep 25 01:01:30.510898 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 346/682] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:30 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 439 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:30.534537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] validate_networks() for [('5dc49b54-686c-4619-ad4c-65c72c26fd31', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:01:30.700264 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:30.701195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:30.708768 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:30.709345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:30.709996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:30.710828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:30.711292 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:30.711707 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:30.750942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=97,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:30.871206 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:30.886685 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4e94e54e-2c87-47fe-a9e0-4d490cb4c014 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1200 microversion: 2.1 time: 0.191245 Sep 25 01:01:30.888904 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 347/683] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:30 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1200 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:31.565506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:31.566837 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:31.578978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:31.580252 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:31.581151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:31.582483 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:31.584597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:31.585550 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:31.810897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:01:31.811658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:31.811847 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:01:31.813798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:01:31.813798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:01:31.814378 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:01:31.823973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:31.826045 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:31.828793 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-43f86e55-3fd2-48d5-9207-38d958628af9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.274646 Sep 25 01:01:31.830526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 348/684] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:31 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 278 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:31.851190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:31.852119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:31.854010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:31.855345 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:31.855703 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:31.856153 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:31.881658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:01:31.887740 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Getting quotas for user 96386a16f0b54b7c94d32bd948caf118 and project f08425c00f0c49ddbe42b21ddbc7ff3f. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:01:31.908559 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:01:31.916241 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:31.917901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:01:31.918548 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:31.940577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:31.943986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:31.945884 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:31.948508 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:31.949334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:31.951559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:31.987553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7d7ddf2c-3cda-447d-b310-f29e0fdc8c82 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 401 microversion: 2.1 time: 1.916902 Sep 25 01:01:31.988636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 337/685] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:01:30 2022] POST /compute/v2.1/servers => generated 401 bytes in 1919 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 01:01:32.004166 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ed4e9d40-5ee9-42cb-ada2-b495af61744c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:32.004829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ed4e9d40-5ee9-42cb-ada2-b495af61744c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:32.085582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:32.104115 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0c6df29a-c93e-46bb-988a-eaefeb5e983f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1210 microversion: 2.1 time: 0.192180 Sep 25 01:01:32.105055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 349/686] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:31 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1210 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:32.142805 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ed4e9d40-5ee9-42cb-ada2-b495af61744c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:32.166842 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ed4e9d40-5ee9-42cb-ada2-b495af61744c tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1241 microversion: 2.1 time: 0.165552 Sep 25 01:01:32.167429 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 338/687] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:31 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1241 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:32.888394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:32.890211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:32.902733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:32.909113 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:32.910013 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:32.912437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:32.913490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:32.914221 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:33.141322 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:33.143042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:33.155127 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:33.162274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:33.164502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:33.168905 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:33.172964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:33.174095 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:33.349873 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:33.368007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:33.371120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:33.374799 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-36b5f2b6-f9c0-4e76-8e90-bab0256f9d16 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.494973 Sep 25 01:01:33.375386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 350/688] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:32 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:33.403252 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:33.406488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:33.425011 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:33.427235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:33.427615 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:33.430579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:33.432216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:33.436218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:33.580972 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:33.595824 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-72aee511-970b-4060-ba6d-d1bb2551daee tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1293 microversion: 2.1 time: 0.217092 Sep 25 01:01:33.596372 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 351/689] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:33 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1293 bytes in 219 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:33.655574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:33.677524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-15ba43d0-0981-452c-b43f-7d4e1cd22a21 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.540574 Sep 25 01:01:33.679651 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 339/690] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:33 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 544 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:34.432118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:34.440580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:34.462063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:34.466307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:34.467629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:34.469191 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:34.470706 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:34.471652 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:34.626451 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:34.628075 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:34.645013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:34.645584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:34.646027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:34.646755 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:34.647216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:34.647650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:34.850694 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=4 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:34.890196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:34.904495 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:34.904872 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:34.906394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-47b77f7c-3484-4159-8006-e66006445b98 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.502620 Sep 25 01:01:34.907721 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 352/691] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:34 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 506 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:34.913926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:34.914874 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:34.922811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:34.923584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:34.924219 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:34.925120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:34.925768 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:34.926354 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:35.014392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:35.032206 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1745c50e-3e05-498c-aad1-2eaed7b8e104 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1299 microversion: 2.1 time: 0.413033 Sep 25 01:01:35.033400 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 340/692] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:34 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1299 bytes in 415 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:35.293558 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:35.306352 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-26572306-9c53-4e04-9a09-9f67e76a7405 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.395861 Sep 25 01:01:35.307877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 353/693] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:34 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 399 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:35.861106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-d5d46174-2cad-45b6-b2e9-5e2956edd7ce req-18a53a69-ef58-4b81-9d6d-13ec7aef7e1f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fdb14c73-6de8-4097-83d7-44f81a01abab", "tag": "b7527175-254d-4003-8922-2e8834ba4a91"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:35.886106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-d5d46174-2cad-45b6-b2e9-5e2956edd7ce req-18a53a69-ef58-4b81-9d6d-13ec7aef7e1f service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:35.887545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-d5d46174-2cad-45b6-b2e9-5e2956edd7ce req-18a53a69-ef58-4b81-9d6d-13ec7aef7e1f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:35.888699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-d5d46174-2cad-45b6-b2e9-5e2956edd7ce req-18a53a69-ef58-4b81-9d6d-13ec7aef7e1f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:35.909850 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-d5d46174-2cad-45b6-b2e9-5e2956edd7ce req-18a53a69-ef58-4b81-9d6d-13ec7aef7e1f service nova] Creating event network-changed:b7527175-254d-4003-8922-2e8834ba4a91 for instance fdb14c73-6de8-4097-83d7-44f81a01abab on hv-cinder-maste Sep 25 01:01:35.918809 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-d5d46174-2cad-45b6-b2e9-5e2956edd7ce req-18a53a69-ef58-4b81-9d6d-13ec7aef7e1f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063167 Sep 25 01:01:35.920158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 341/694] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:35 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:35.961164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:35.969677 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:35.999915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:36.004146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:36.007471 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:36.017820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:36.018643 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:36.019363 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:36.086023 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:36.104330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:36.149115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:36.154877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:36.158410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:36.159815 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:36.160697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:36.161544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:36.324248 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:36.350216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:36.353654 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:36.381465 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2ecf0e9d-ffb1-4013-a473-80e76d2bfb6c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.423837 Sep 25 01:01:36.387847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 354/695] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:35 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 436 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:36.414755 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:36.418112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:36.422667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:36.426732 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d8d01bd0-0614-48a2-a2d3-b8a207f99618 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1299 microversion: 2.1 time: 0.357964 Sep 25 01:01:36.429682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 342/696] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:36 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1299 bytes in 362 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:36.454737 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:36.458832 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:36.461916 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:36.462975 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:36.464326 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:36.465827 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:36.946532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:36.972587 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4c94fe4d-72c3-4c20-af09-f28fa99f5d9f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.582783 Sep 25 01:01:36.975504 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 355/697] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:36 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:37.425528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:37.428452 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:37.444042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:37.445025 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:37.446151 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:37.449220 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:37.450199 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:37.451270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:37.476437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:37.487116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:37.515159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:37.516524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:37.517683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:37.519416 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:37.520534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:37.521702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:37.873287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:37.888121 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-43b3c9d0-9c33-4964-9d30-ac53df9173d4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.423815 Sep 25 01:01:37.901749 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 356/698] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:37 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 433 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:37.974231 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:38.000386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:38.003638 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:38.005344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-66fdad7a-63d1-45bd-abf3-badae90dbc62 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.593765 Sep 25 01:01:38.006459 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 343/699] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:37 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 596 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:38.033224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:38.034133 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:38.058946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:38.059451 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:38.059892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:38.060626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:38.061125 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:38.061547 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:38.356294 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:38.367594 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0d797822-3311-4338-92f5-b3c3db076899 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.347945 Sep 25 01:01:38.369576 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 357/700] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:38 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 351 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:38.913722 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:38.915921 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:38.924181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:38.924674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:38.925313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:38.926236 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:38.926918 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:38.927528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:39.053314 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:39.054857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:39.072209 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:39.073178 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:39.075955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:39.077176 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:39.078062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:39.082995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:39.413800 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:39.430420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:39.431164 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ee9b0a74-6722-4e11-8548-066460954db4 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.521230 Sep 25 01:01:39.433994 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 344/701] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:38 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 526 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:39.450169 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:39.451140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:39.454690 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:39.456705 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:39.458250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-df80a197-42c6-4386-bef0-186718ab75aa tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.410197 Sep 25 01:01:39.473858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:39.473858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:39.473858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:39.473858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:39.474167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:39.474167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:39.474862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 358/702] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:39 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 427 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:01:39.743751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-9e58e512-e1d3-47ed-811a-adfb922a4510 req-8e049821-e898-451c-b32a-b0174755e580 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb07cc70-0299-47dc-97cb-e3b82c1a47b2", "tag": "e90855c6-13bf-47db-8fb7-7226a222a0c7"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:39.757896 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-9e58e512-e1d3-47ed-811a-adfb922a4510 req-8e049821-e898-451c-b32a-b0174755e580 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:39.758394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-9e58e512-e1d3-47ed-811a-adfb922a4510 req-8e049821-e898-451c-b32a-b0174755e580 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:39.759735 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-9e58e512-e1d3-47ed-811a-adfb922a4510 req-8e049821-e898-451c-b32a-b0174755e580 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:39.789633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-9e58e512-e1d3-47ed-811a-adfb922a4510 req-8e049821-e898-451c-b32a-b0174755e580 service nova] Creating event network-changed:e90855c6-13bf-47db-8fb7-7226a222a0c7 for instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 on hv-cinder-maste Sep 25 01:01:39.802996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-9e58e512-e1d3-47ed-811a-adfb922a4510 req-8e049821-e898-451c-b32a-b0174755e580 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.063684 Sep 25 01:01:39.805337 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 359/703] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:39 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:01:40.011280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:40.031148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d53bc417-db08-48e5-9124-a71fbe49a674 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.595875 Sep 25 01:01:40.033438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 345/704] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:39 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 599 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:40.462576 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:40.464265 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:40.472046 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:40.472322 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:40.473542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:40.474401 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:40.474876 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:40.475306 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:40.510065 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:40.512388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:40.527623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:40.529298 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:40.530596 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:40.532091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:40.533834 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:40.535085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:40.732432 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:40.744342 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-286a166a-71d6-4591-aa7c-ea63058e2ab7 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.287815 Sep 25 01:01:40.758228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 360/705] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:40 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:41.066711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:41.068092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:41.070608 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:41.075621 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:41.082272 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:41.084270 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:41.085173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:41.086096 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:41.086253 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-adbbc01e-e623-4c59-9e05-e2cccea55ccf tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1745 microversion: 2.20 time: 0.602898 Sep 25 01:01:41.087210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 346/706] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:40 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1745 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:41.101724 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:41.107635 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:41.108650 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:41.110028 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:41.353305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:41.370440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-14da3810-54a3-4c0c-a58c-8b705c8f2b24 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.310291 Sep 25 01:01:41.371512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 361/707] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:41 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 313 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:41.779404 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:41.780483 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:41.788722 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:41.789681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:41.790784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:41.791527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:41.791990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:41.792416 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:41.867454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=4 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:42.104874 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:42.120893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4f0c5dbe-d4bb-4565-92e2-8c0faf3652e1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.353849 Sep 25 01:01:42.121908 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 347/708] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:41 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 356 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:42.140948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:42.141816 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:42.151632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:42.152202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:42.152644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:42.153432 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:42.154608 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:42.155091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:42.410713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:42.411736 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:42.418786 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=16 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:42.442580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:42.447367 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:42.448490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:42.449843 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:42.450718 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:42.451480 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:42.699573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:42.719795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:42.721733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:42.723320 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-390c13c6-435c-4f15-9f46-1a6a5b0ff6e9 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1745 microversion: 2.20 time: 0.591676 Sep 25 01:01:42.724582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 362/709] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:42 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1745 bytes in 594 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:42.782027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:42.797739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-71e0188f-48e7-4cb6-8bea-8269200d3492 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.393654 Sep 25 01:01:42.809378 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 348/710] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:42 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 396 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:43.056141 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-1a12f0b5-cd2d-4d13-b98a-a508d4ef6ebe req-157b8d7f-a16e-4e48-b542-1008b646c0c6 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4", "name": "network-vif-plugged", "status": "completed", "tag": "4aef1009-d2d3-4c6f-af8a-c5134e19fa37"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:43.066787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1a12f0b5-cd2d-4d13-b98a-a508d4ef6ebe req-157b8d7f-a16e-4e48-b542-1008b646c0c6 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:43.067272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1a12f0b5-cd2d-4d13-b98a-a508d4ef6ebe req-157b8d7f-a16e-4e48-b542-1008b646c0c6 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:43.067698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-1a12f0b5-cd2d-4d13-b98a-a508d4ef6ebe req-157b8d7f-a16e-4e48-b542-1008b646c0c6 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:43.080950 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-1a12f0b5-cd2d-4d13-b98a-a508d4ef6ebe req-157b8d7f-a16e-4e48-b542-1008b646c0c6 service nova] Creating event network-vif-plugged:4aef1009-d2d3-4c6f-af8a-c5134e19fa37 for instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 on hv-cinder-maste Sep 25 01:01:43.089280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-1a12f0b5-cd2d-4d13-b98a-a508d4ef6ebe req-157b8d7f-a16e-4e48-b542-1008b646c0c6 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.362820 Sep 25 01:01:43.090151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 363/711] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:42 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 365 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:43.149119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:43.150560 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:43.158933 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:43.164550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:43.164550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:43.164550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:43.164550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:43.166608 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:43.453175 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:43.470534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8719289a-60ce-4bd6-b78c-0ac8cd2c709b tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.326726 Sep 25 01:01:43.471610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 349/712] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:43 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 329 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:43.765868 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:43.767157 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:43.775682 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:43.776422 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:43.777019 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:43.777957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:43.778412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:43.778842 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:43.844159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:43.845054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:43.866088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:43.867757 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:43.870596 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:43.873152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:43.874812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:43.877478 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:44.103494 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:44.127116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:44.132443 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:44.149512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-21915323-1cd7-4b1e-81d3-f7e5f31ec78e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.384606 Sep 25 01:01:44.154667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 364/713] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:43 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 395 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:01:44.499956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:44.501061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:44.527053 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:44.533060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:44.536606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:44.540272 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:44.542803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:44.545664 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:44.647553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:44.651949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=100 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:44.683795 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3415767a-ca02-4743-a4ba-92d5071bdade tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.834996 Sep 25 01:01:44.685157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 350/714] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:43 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 849 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:45.060565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:45.090858 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1f420546-8e2b-4541-8942-6302f710b422 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.598000 Sep 25 01:01:45.096622 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 365/715] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:44 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 605 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:45.192200 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:45.198275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:45.222875 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:45.230430 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:45.232003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:45.240315 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:45.242218 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.007s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:45.244587 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:45.732663 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:45.733939 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:45.783823 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:45.784317 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:45.788130 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:45.791285 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:45.792166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:45.793822 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:46.141211 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:46.169877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:46.175910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:46.185113 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:46.188103 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4bc70970-3a45-48b5-9076-9d3d8a54f565 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 1.003287 Sep 25 01:01:46.191057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 351/716] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:45 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 1017 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:46.197534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-dc81cf67-85b3-4e6a-bd9e-04018ce0022a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.474797 Sep 25 01:01:46.205701 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 366/717] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:45 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 484 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:46.211140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:46.218320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:46.229223 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:46.230304 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:46.230978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:46.231977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:46.232657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:46.233292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:46.635626 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:46.646540 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a5cb85d2-2505-4a4e-821e-106f2417b473 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.453549 Sep 25 01:01:46.662710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 352/718] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:46 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 471 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:47.220412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:47.221021 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:47.231338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:47.231838 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:47.232428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:47.233421 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:47.234051 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:47.234690 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:47.245287 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:47.246610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:47.260140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:47.267465 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:47.269598 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:47.270840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:47.276719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:47.277180 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:47.543597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:47.557710 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:47.559306 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:47.566548 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-51bcc367-247f-4c2d-b170-f1311310a80c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.346295 Sep 25 01:01:47.568819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 367/719] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:47 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 353 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:01:47.679527 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:47.682758 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:47.712200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:47.714775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:47.725172 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:47.727590 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:47.728938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:47.730277 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:47.993467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:48.006571 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-05376aee-ae36-4b24-845e-8bfe1c0dbc0b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.769364 Sep 25 01:01:48.023639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 353/720] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:47 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 787 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:48.100568 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:48.152629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c5590541-dcc4-4315-a472-9bad9b76933e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.481150 Sep 25 01:01:48.156669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 368/721] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:47 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 487 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:48.601883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:48.604670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:48.624709 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:48.625781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:48.629531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:48.635318 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:48.636364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:48.640037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:49.044624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:49.046165 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:49.056718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:49.059359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:49.060504 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:49.062056 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:49.063202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:49.065765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:49.070843 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:49.091890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:49.095320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:49.099986 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-225ef428-6c7a-46aa-bd7c-6cd57c0b5be2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.504369 Sep 25 01:01:49.145530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 354/722] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:48 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 553 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:49.179838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:49.181379 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:49.192896 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:49.200703 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:49.201916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:49.202914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:49.203591 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:49.204371 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:49.588921 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:49.631538 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5fe84344-2494-416d-9a1d-b1481f84b301 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.456308 Sep 25 01:01:49.637008 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 355/723] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:49 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 463 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:49.881768 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:49.890809 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-80ce2a73-50ef-4e02-bd4b-38090b32dfd2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.853801 Sep 25 01:01:49.902644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 369/724] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:49 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 859 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:50.165964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:50.171380 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:50.200340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:50.204532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:50.207373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:50.212150 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:50.216522 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:50.218204 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:50.618399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-2ca60e41-8b5b-4cdd-ae24-ac28d4d1c3e9 req-445d7353-182c-4488-989b-b11252b1e52e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb07cc70-0299-47dc-97cb-e3b82c1a47b2", "name": "network-vif-plugged", "status": "completed", "tag": "e90855c6-13bf-47db-8fb7-7226a222a0c7"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:50.652457 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2ca60e41-8b5b-4cdd-ae24-ac28d4d1c3e9 req-445d7353-182c-4488-989b-b11252b1e52e service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:50.653334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2ca60e41-8b5b-4cdd-ae24-ac28d4d1c3e9 req-445d7353-182c-4488-989b-b11252b1e52e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:50.654004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2ca60e41-8b5b-4cdd-ae24-ac28d4d1c3e9 req-445d7353-182c-4488-989b-b11252b1e52e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:50.687154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:50.714920 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:50.719736 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-2ca60e41-8b5b-4cdd-ae24-ac28d4d1c3e9 req-445d7353-182c-4488-989b-b11252b1e52e service nova] Creating event network-vif-plugged:e90855c6-13bf-47db-8fb7-7226a222a0c7 for instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 on hv-cinder-maste Sep 25 01:01:50.732949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:50.739325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7c857367-f2cf-4441-bbc6-e5cd73b0ea5c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.582126 Sep 25 01:01:50.741781 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-2ca60e41-8b5b-4cdd-ae24-ac28d4d1c3e9 req-445d7353-182c-4488-989b-b11252b1e52e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.125213 Sep 25 01:01:50.744515 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 356/725] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:50 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 590 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:50.747535 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 370/726] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 135 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:50.754893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:50.756531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:50.769392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:50.770896 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:50.772094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:50.774498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:50.775928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:50.777086 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:50.976384 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:50.979486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:51.007114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:51.009064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:51.010358 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:51.026511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:51.027944 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:51.031731 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:51.315943 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:51.352860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7fb83ef5-7afe-46c9-b6ed-00cb717eb9dc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.604748 Sep 25 01:01:51.358008 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 357/727] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:50 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 612 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:51.621514 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:51.630347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7c07ae77-90a9-4ed4-9136-84aeb9a2ab22 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.665760 Sep 25 01:01:51.630930 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 371/728] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:50 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 687 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:51.797853 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:51.798943 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:51.818138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:51.819098 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:51.819929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:51.821389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:51.822187 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:51.822938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:52.334215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:52.362799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:52.365997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:52.368237 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a4398594-f616-49eb-b800-8b7fe661255d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.577602 Sep 25 01:01:52.376624 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 358/729] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:51 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 587 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:52.389672 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:52.395266 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:52.421532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=86,nova_api:DELETE=1,nova_api:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:52.427010 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:52.428963 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:52.430867 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:52.432893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:52.434466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:52.435532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:52.663744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:52.664929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:52.689375 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=93,nova_api:DELETE=1,nova_api:INSERT=6 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:52.699419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:52.700297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:52.701510 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:52.703171 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:52.704345 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:52.705619 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:53.142644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:53.172036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cbbb0e12-5e78-485b-b379-6522d9ab1638 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1366 microversion: 2.1 time: 0.787936 Sep 25 01:01:53.177797 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 372/730] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:52 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1366 bytes in 798 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:53.202625 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-15e3cf8f-e0e4-4ea3-a2df-f3b91edd2886 req-4ba2e1ac-4839-4404-bd22-32fcf694b8ea service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdb14c73-6de8-4097-83d7-44f81a01abab", "name": "network-vif-plugged", "status": "completed", "tag": "b7527175-254d-4003-8922-2e8834ba4a91"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:53.244287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-15e3cf8f-e0e4-4ea3-a2df-f3b91edd2886 req-4ba2e1ac-4839-4404-bd22-32fcf694b8ea service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:53.248437 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-15e3cf8f-e0e4-4ea3-a2df-f3b91edd2886 req-4ba2e1ac-4839-4404-bd22-32fcf694b8ea service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:53.251598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-15e3cf8f-e0e4-4ea3-a2df-f3b91edd2886 req-4ba2e1ac-4839-4404-bd22-32fcf694b8ea service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:53.295819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-15e3cf8f-e0e4-4ea3-a2df-f3b91edd2886 req-4ba2e1ac-4839-4404-bd22-32fcf694b8ea service nova] Creating event network-vif-plugged:b7527175-254d-4003-8922-2e8834ba4a91 for instance fdb14c73-6de8-4097-83d7-44f81a01abab on hv-cinder-maste Sep 25 01:01:53.319051 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-15e3cf8f-e0e4-4ea3-a2df-f3b91edd2886 req-4ba2e1ac-4839-4404-bd22-32fcf694b8ea service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.138299 Sep 25 01:01:53.324087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:53.324992 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 373/731] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:53 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 146 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:01:53.382671 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-61fa4d5f-28d2-4145-907c-3fc49a8f5d73 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.725590 Sep 25 01:01:53.383789 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 359/732] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:52 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 728 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:01:53.423943 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:53.427164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:53.456845 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:53.466096 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:53.467343 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:53.468898 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:53.470027 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:53.475782 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:54.070179 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:54.115965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:54.137579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:54.141792 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e2befd0d-4377-4430-a0a4-3742b7bcb149 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.724324 Sep 25 01:01:54.146097 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 374/733] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:53 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 732 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:54.228410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:54.230091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:54.244299 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:54.245566 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:54.246584 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:54.250023 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:54.252510 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:54.253661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:54.427025 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:54.449771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:54.483066 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:54.486996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:54.488191 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:54.491339 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:54.492205 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:54.493079 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:54.740606 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:54.772221 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-aad6b0c0-a909-4564-a79c-470c12d16dfa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1556 microversion: 2.1 time: 0.537866 Sep 25 01:01:54.776264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 360/734] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:54 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1556 bytes in 558 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:54.808219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:54.810374 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:54.846525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:54.850449 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:54.851651 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:54.855117 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:54.860757 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:54.865641 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:55.025492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:55.069817 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f0e9c8a5-353f-4b1c-8304-1b5cbcdbb329 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1266 microversion: 2.1 time: 0.648293 Sep 25 01:01:55.094391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 375/735] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:54 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1266 bytes in 677 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:55.197063 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4267cff0-96e6-49f6-a2ff-d7c1df9207bc tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.401243 Sep 25 01:01:55.198096 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 361/736] 10.222.0.52 () {58 vars in 1275 bytes} [Sun Sep 25 01:01:54 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-interface => generated 285 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:01:55.208319 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:55.212772 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:55.248556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:55.256088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:55.257743 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:55.264045 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:55.268879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:55.270266 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:55.990858 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:56.005055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:56.012145 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:56.017372 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1e58cdf9-2762-4242-bf42-7b11bfb97acc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.820871 Sep 25 01:01:56.029716 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 376/737] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:55 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 838 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:56.139450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:56.143536 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:56.187393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:56.188263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:56.196734 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:56.204242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:56.207777 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:56.210526 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:56.647398 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:56.657059 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-74d1c380-372c-4fd0-ae6a-7c0d8286f9ee tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1453 microversion: 2.1 time: 0.543023 Sep 25 01:01:56.658220 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 362/738] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:56 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1453 bytes in 545 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:56.712906 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:56.726433 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:56.752383 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:56.764362 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:56.767005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:56.769184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:56.772195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:56.778008 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:57.049105 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:57.051062 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:57.095514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:57.102034 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:57.107034 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:57.108460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:57.111246 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:57.112143 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:57.583338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:57.591979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=100 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:01:57.614876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:57.617088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:57.622728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5a2a6462-9023-4cd6-bafc-e8805cecd4f0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.580263 Sep 25 01:01:57.655216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 363/739] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:57 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 615 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:58.103177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:58.162801 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-88cc5e02-1887-4aaa-a39f-6b30e55d4340 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1453 microversion: 2.1 time: 1.455570 Sep 25 01:01:58.172867 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 377/740] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:01:56 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1453 bytes in 1470 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:01:58.675067 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:01:58.675630 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:01:58.694282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:58.698223 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:58.698737 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:58.699513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:58.699996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:58.700446 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:59.106385 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:59.127135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:59.145600 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:01:59.155321 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9ce7e0c4-df12-4fec-a59d-1a3a55a8b9d6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.481010 Sep 25 01:01:59.162294 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 364/741] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:01:58 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 492 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:01:59.537793 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-ebefdded-f982-4247-a9af-956ac90f7fd9 req-03502b61-d8cf-43f5-9285-416d6170ad02 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "cb07cc70-0299-47dc-97cb-e3b82c1a47b2", "tag": "e90855c6-13bf-47db-8fb7-7226a222a0c7"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:01:59.578952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-ebefdded-f982-4247-a9af-956ac90f7fd9 req-03502b61-d8cf-43f5-9285-416d6170ad02 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:01:59.582063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-ebefdded-f982-4247-a9af-956ac90f7fd9 req-03502b61-d8cf-43f5-9285-416d6170ad02 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:01:59.585587 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-ebefdded-f982-4247-a9af-956ac90f7fd9 req-03502b61-d8cf-43f5-9285-416d6170ad02 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:01:59.619808 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-ebefdded-f982-4247-a9af-956ac90f7fd9 req-03502b61-d8cf-43f5-9285-416d6170ad02 service nova] Creating event network-changed:e90855c6-13bf-47db-8fb7-7226a222a0c7 for instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 on hv-cinder-maste Sep 25 01:01:59.637704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-ebefdded-f982-4247-a9af-956ac90f7fd9 req-03502b61-d8cf-43f5-9285-416d6170ad02 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.102809 Sep 25 01:01:59.639769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 378/742] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:01:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 110 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:02:00.186956 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:00.191506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:00.216490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:00.220936 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:00.225489 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:00.232690 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:00.239585 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:00.240710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:01.078142 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:01.152285 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:01.158234 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:01.162360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-254ce77a-ff29-418f-9d71-fdaf7a16b980 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.983639 Sep 25 01:02:01.162990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 365/743] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:00 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 985 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:02.242779 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:02.244177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:02.313914 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:02.319308 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:02.323072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:02.331338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:02.339277 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:02.344341 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:03.139089 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:03.153799 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:03.155324 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:03.156562 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1c63fb98-9f14-46cc-bfca-23755acd741a tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.963509 Sep 25 01:02:03.173754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 379/744] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:02 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 985 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:03.823534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-5b77e5b8-0e21-485c-89db-306f1c927282 req-71cb9976-51f0-4acb-9666-994e3f19b69f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fdb14c73-6de8-4097-83d7-44f81a01abab", "tag": "b7527175-254d-4003-8922-2e8834ba4a91"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:03.865055 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-5b77e5b8-0e21-485c-89db-306f1c927282 req-71cb9976-51f0-4acb-9666-994e3f19b69f service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:03.866165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-5b77e5b8-0e21-485c-89db-306f1c927282 req-71cb9976-51f0-4acb-9666-994e3f19b69f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:03.867436 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-5b77e5b8-0e21-485c-89db-306f1c927282 req-71cb9976-51f0-4acb-9666-994e3f19b69f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:03.923308 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-5b77e5b8-0e21-485c-89db-306f1c927282 req-71cb9976-51f0-4acb-9666-994e3f19b69f service nova] Creating event network-changed:b7527175-254d-4003-8922-2e8834ba4a91 for instance fdb14c73-6de8-4097-83d7-44f81a01abab on hv-cinder-maste Sep 25 01:02:03.932350 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-5b77e5b8-0e21-485c-89db-306f1c927282 req-71cb9976-51f0-4acb-9666-994e3f19b69f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.113126 Sep 25 01:02:03.941422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 366/745] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:02:03 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:02:04.184538 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:04.185624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:04.206408 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:04.214984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:04.217647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.010s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:04.218607 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:04.219215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:04.222852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:04.767979 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:04.793491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:04.802162 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:04.809969 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a0a6956a-d64c-4dc8-890f-a597b59424e6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.625028 Sep 25 01:02:04.811532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 380/746] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:04 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 633 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:05.843018 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:05.845406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:05.865275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:05.869877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:05.870748 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:05.880484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:05.881391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:05.884766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:06.719576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:06.760352 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:06.776431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:06.784248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e2a2a158-f55c-4285-a9d9-51624f8440e7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.946718 Sep 25 01:02:06.785155 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 367/747] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:05 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 949 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:07.832820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:07.833850 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:07.855978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:07.860612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:07.861315 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:07.862490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:07.863357 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:07.864239 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:08.342317 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:08.372009 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:08.376634 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:08.379590 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4ec3f9ea-fb86-4771-9a64-3fa9d64d1af6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.551328 Sep 25 01:02:08.381771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 381/748] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:07 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 554 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:09.422946 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:09.424983 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:09.445627 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:09.465863 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.023s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:09.479551 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.014s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:09.482735 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:09.483590 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:09.484325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:10.014367 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:10.036123 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:10.039995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:10.042210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4209b25d-8fb3-47ac-a698-318f743d6758 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.628951 Sep 25 01:02:10.043405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 368/749] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:09 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 631 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:11.074128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:11.074763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:11.092761 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:11.093964 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.008s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:11.094745 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:11.096003 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:11.096804 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:11.097617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:11.398890 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:11.417633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:11.419571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:11.421347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1b3d026b-b714-46d9-b967-09c5e76c5c0d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.353100 Sep 25 01:02:11.456672 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 382/750] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:11 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 389 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:12.455039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:12.455859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:12.470597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:12.471274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:12.476367 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:12.486059 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:12.486908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.009s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:12.487485 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:12.678165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e6444a3f-18c7-4080-a8e5-c552e72b2496 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Action: 'create', calling method: .version_select at 0x7fe22d77fb80>, body: {"keypair": {"name": "tempest-TestShelveInstance-102314433", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBD0Jmfj+a7ZTaNNEFRoWC+fy8uTt6kthVznRNRw5AzCLxeY8Vn5m29Ysi9depjcNIihXZdl3TSuGLgh9ECGq1vDaDZjxf8wMJoPGyaa1o5UKMfLhEpy8y2OBLFJBsSLM5A=="}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:12.740206 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-e6444a3f-18c7-4080-a8e5-c552e72b2496 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Getting quotas for project 69a3d6ce251e480d866a86ed9f6f2eb9. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:02:12.750129 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-e6444a3f-18c7-4080-a8e5-c552e72b2496 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Getting quotas for user 598ec83f7b7f4a9f90543218852aaf14 and project 69a3d6ce251e480d866a86ed9f6f2eb9. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:02:12.802563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:12.807799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e6444a3f-18c7-4080-a8e5-c552e72b2496 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 396 microversion: 2.1 time: 0.611077 Sep 25 01:02:12.809098 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 369/751] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 01:02:12 2022] POST /compute/v2.1/os-keypairs => generated 396 bytes in 614 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:02:12.825946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:12.829112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:12.832794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-058f2001-8ae8-416b-9c82-04f1cca75652 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.383650 Sep 25 01:02:12.834210 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 383/752] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:12 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 386 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:13.891106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:13.892504 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:13.908554 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:13.909390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:13.910001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:13.910918 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:13.911559 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:13.912338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:14.252863 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:14.283462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:14.306085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:14.308977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ed5dabd4-0521-45b5-b9a9-50c33cec28b6 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1740 microversion: 2.20 time: 0.422130 Sep 25 01:02:14.310396 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 370/753] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:13 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1740 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:15.336702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:15.338036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:15.345674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:15.348431 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:15.348888 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:15.349678 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:15.350152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:15.350588 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:15.376429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=99 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:15.645120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:15.655863 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:15.657769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:15.659276 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f248d0e8-f411-40d0-8366-1b8e68b817b4 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1722 microversion: 2.20 time: 0.327141 Sep 25 01:02:15.669930 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 384/754] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:15 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1722 bytes in 338 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:16.695100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:16.696293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:16.710869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:16.711774 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:16.712561 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:16.713936 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:16.714789 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:16.715563 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:16.950388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:16.960154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:16.962343 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:16.963579 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-77dbc344-0baa-46f0-a58f-be14dbf8fe38 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1722 microversion: 2.20 time: 0.280189 Sep 25 01:02:16.964162 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 371/755] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:16 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1722 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:18.006345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:18.008509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:18.017099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:18.018025 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:18.018887 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:18.019795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:18.020404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:18.020966 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:18.223887 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:18.233373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:18.235535 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:18.237000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8b044d91-e406-41ce-8e31-5d7a50f68f08 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1725 microversion: 2.20 time: 0.235569 Sep 25 01:02:18.238166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 385/756] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:18 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1725 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:18.294841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "35e505eb-c980-4cb6-8d83-8f9b713f8333"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:18.299346 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:18.320816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:18.321633 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:18.322433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:18.324013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:18.324895 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:18.325904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:18.962730 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-91823b26-0e32-4338-b99a-12b0fc811a40 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/os-volume_attachments" status: 200 len: 188 microversion: 2.20 time: 0.680114 Sep 25 01:02:18.963883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 372/757] 10.222.0.52 () {62 vars in 1364 bytes} [Sun Sep 25 01:02:18 2022] POST /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/os-volume_attachments => generated 188 bytes in 683 msecs (HTTP/1.1 200) 9 headers in 359 bytes (1 switches on core 0) Sep 25 01:02:19.169181 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:19.170614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:19.179518 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:19.180767 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:19.181543 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:19.182941 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:19.183647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:19.184208 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:19.692687 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0b833342-fdee-4609-a1e9-761cf05ca0c8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "DELETE /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/os-volume_attachments/35e505eb-c980-4cb6-8d83-8f9b713f8333" status: 202 len: 0 microversion: 2.20 time: 0.527532 Sep 25 01:02:19.694117 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 386/758] 10.222.0.52 () {60 vars in 1457 bytes} [Sun Sep 25 01:02:19 2022] DELETE /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/os-volume_attachments/35e505eb-c980-4cb6-8d83-8f9b713f8333 => generated 0 bytes in 531 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:02:19.839502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:19.840326 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:19.847521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:19.848073 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:19.848518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:19.849250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:19.849757 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:19.850185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:19.918770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120450) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 25 01:02:20.050937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-e31dc611-7df9-4ebe-adb2-bb512a10f73d req-5a8e599e-63ca-4f8e-8f6b-c3b3e274a47c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4", "name": "network-vif-unplugged", "status": "completed", "tag": "4aef1009-d2d3-4c6f-af8a-c5134e19fa37"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:20.074846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f6f6d820-90db-4f33-ac49-3a3e749d852b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "POST /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/action" status: 202 len: 0 microversion: 2.20 time: 0.228911 Sep 25 01:02:20.076355 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 373/759] 10.222.0.52 () {62 vars in 1319 bytes} [Sun Sep 25 01:02:19 2022] POST /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/action => generated 0 bytes in 242 msecs (HTTP/1.1 202) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:02:20.089283 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-e31dc611-7df9-4ebe-adb2-bb512a10f73d req-5a8e599e-63ca-4f8e-8f6b-c3b3e274a47c service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:20.091864 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:20.093281 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-e31dc611-7df9-4ebe-adb2-bb512a10f73d req-5a8e599e-63ca-4f8e-8f6b-c3b3e274a47c service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:20.093553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:20.096618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-e31dc611-7df9-4ebe-adb2-bb512a10f73d req-5a8e599e-63ca-4f8e-8f6b-c3b3e274a47c service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:20.109222 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:20.111506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:20.113431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:20.115850 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:20.117888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:20.119184 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:20.136273 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.compute.server_external_events [req-e31dc611-7df9-4ebe-adb2-bb512a10f73d req-5a8e599e-63ca-4f8e-8f6b-c3b3e274a47c service nova] Unable to find a host for instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4. Dropping event network-vif-unplugged {{(pid=120451) create /opt/stack/nova/nova/api/openstack/compute/server_external_events.py:144}} Sep 25 01:02:20.138258 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-e31dc611-7df9-4ebe-adb2-bb512a10f73d req-5a8e599e-63ca-4f8e-8f6b-c3b3e274a47c service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 207 len: 182 microversion: 2.1 time: 0.093336 Sep 25 01:02:20.139243 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 387/760] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:02:20 2022] POST /compute/v2.1/os-server-external-events => generated 182 bytes in 96 msecs (HTTP/1.1 207) 9 headers in 367 bytes (2 switches on core 0) Sep 25 01:02:20.376262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:20.386023 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:20.387526 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:20.388762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8faa026c-ac50-41a0-92f8-172b9c3a6dc8 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1673 microversion: 2.20 time: 0.307283 Sep 25 01:02:20.413154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 374/761] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:20 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1673 bytes in 333 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:21.422005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:21.423366 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:21.431597 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:21.432547 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:21.433140 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:21.434098 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:21.434759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:21.435412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:21.627624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:21.638399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:21.640371 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:21.641978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-369704a1-7195-4aa6-9511-ee8064a9e641 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1673 microversion: 2.20 time: 0.224723 Sep 25 01:02:21.642983 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 388/762] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:21 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1673 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:22.692051 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:22.693244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:22.706916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:22.707769 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:22.709014 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:22.714368 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:22.715280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:22.716118 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:22.773777 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=4 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:23.032773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:23.050308 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:23.053505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:23.055771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4287dc84-18c3-4e01-986d-9b90d813096d tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.370582 Sep 25 01:02:23.056816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 375/763] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:22 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 373 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:24.090297 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:24.091957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:24.095544 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=34 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:24.103959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:24.104720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:24.105575 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:24.106639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:24.107445 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:24.108525 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:24.310988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:24.341921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:24.344766 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:24.347273 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4fdb14fc-8dc2-4a62-b362-11df54df4679 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.263403 Sep 25 01:02:24.348509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 389/764] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:24 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:25.378341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:25.379340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:25.387794 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=47,nova_api:INSERT=1,nova_api:UPDATE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:25.390449 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:25.390953 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:25.392771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:25.394308 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:25.395222 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:25.396143 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:25.659710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:25.681292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:25.685140 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:25.688655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-69814dd9-7a18-4b6c-aed0-23b7ebdfb964 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.314221 Sep 25 01:02:25.690686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 376/765] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:25 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 317 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:26.738049 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:26.740694 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:26.765053 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:26.765592 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:26.766038 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:26.766785 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:26.767249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:26.767679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:27.017791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:27.027946 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:27.029462 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:27.030833 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-dfbcc2d3-6ae3-4281-b522-f9c53f1b64fd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.301737 Sep 25 01:02:27.031367 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 390/766] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:26 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 303 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:28.061408 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:28.063534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:28.076726 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:28.077665 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:28.078997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:28.080620 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:28.082108 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:28.083162 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:28.093149 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=82,nova_cell1:INSERT=4,nova_cell1:UPDATE=5 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:28.725628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:28.753103 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:28.765784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:28.768342 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-066bb477-6e77-436c-a7e0-2f0f28633638 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.713477 Sep 25 01:02:28.785779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 377/767] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:28 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 733 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:29.813636 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:29.814567 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:29.824741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:29.825988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:29.827022 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:29.828506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:29.829629 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:29.830602 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:30.146250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:30.198231 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:30.202340 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:30.204675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a5aa8670-9bd1-4955-9c00-61438111e1a0 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.395252 Sep 25 01:02:30.206709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 391/768] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:29 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 398 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:31.259038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:31.261407 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:31.275369 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:31.276903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:31.278159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:31.280089 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:31.281314 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:31.282585 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:31.587760 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:31.603589 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:31.608138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:31.610609 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-96a00ba4-c9a0-4b83-ac66-69e3a4a66ed7 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.360197 Sep 25 01:02:31.625781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 378/769] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:31 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 377 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:32.656805 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:32.657565 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:32.665639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:32.666131 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:32.666566 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:32.667269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:32.667720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:32.668140 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:32.841645 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:32.851591 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:32.853055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:32.854287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-17b3542d-2643-4925-9440-e7703e583ed2 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.202052 Sep 25 01:02:32.854877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 392/770] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:32 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:33.881439 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:33.882163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:33.890253 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:33.890751 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:33.891196 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:33.891919 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:33.892550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:33.893147 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:34.044563 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:34.061931 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:34.065094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:34.067815 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-332a31f4-0956-49af-aa4d-bbc858f20182 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.190180 Sep 25 01:02:34.070594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 379/771] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:33 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:35.115885 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:35.116583 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:35.123973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:35.124528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:35.125014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:35.125865 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:35.126434 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:35.126982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:35.299387 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:35.309951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:35.311435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:35.312653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6a3e341a-b151-4a1c-b59f-b15468f319cc tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.201230 Sep 25 01:02:35.313220 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 393/772] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:35 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:36.346526 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:36.347230 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:36.356848 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:36.357377 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:36.358118 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:36.359239 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:36.360131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:36.361701 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:36.550070 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:36.559572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:36.561110 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:36.562386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bde372c4-503d-42cb-997b-df5d6ff1e27b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.221725 Sep 25 01:02:36.562956 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 380/773] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:36 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:37.593608 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:37.594716 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:37.606455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:37.607258 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:37.610694 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:37.611930 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:37.612915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:37.613580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:37.787216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:37.799973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:37.801432 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:37.803775 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-92ba59ba-fef0-4d5c-ad67-d89afe2eb5cb tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.216769 Sep 25 01:02:37.804183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 394/774] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:37 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 220 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:38.846954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:38.847624 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:38.855513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:38.856144 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:38.856698 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:38.857542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:38.858100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:38.858612 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:39.014594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:39.024231 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:39.025711 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:39.026902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9ab514d5-c608-4f3c-9241-f8adb2561a56 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.183899 Sep 25 01:02:39.027474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 381/775] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:38 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:39.855300 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:39.855926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:39.863412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:39.863904 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:39.864368 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:39.865615 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:39.866114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:39.866540 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:39.892532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c9ddef98-3dee-4e83-b4c5-c0179ca9c7c5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/ips" status: 200 len: 150 microversion: 2.1 time: 0.041506 Sep 25 01:02:39.893132 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 395/776] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 01:02:39 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/ips => generated 150 bytes in 43 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:02:40.055115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:40.056158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:40.067963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:40.069164 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:40.070519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:40.071882 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:40.072663 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:40.073481 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:40.273836 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:40.283823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:40.285617 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:40.287111 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0c122f24-365a-4b60-a1a7-c88ce7c96ef5 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.237278 Sep 25 01:02:40.287813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 382/777] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:40 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 239 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:41.313043 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:41.313882 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:41.323339 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:41.323622 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:41.324138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:41.324982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:41.325481 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:41.326046 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:41.487634 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:41.498366 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:41.499923 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:41.501151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ebad7b0e-63e0-4e28-a753-5f2f981bc636 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.193037 Sep 25 01:02:41.505444 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 396/778] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:41 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 198 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:41.786433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-90936226-be47-42a6-89ec-5b5a8d65bd8c req-170751cc-28ef-4219-aac6-21acd44e2995 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4", "name": "network-vif-plugged", "status": "completed", "tag": "4aef1009-d2d3-4c6f-af8a-c5134e19fa37"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:41.809029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-90936226-be47-42a6-89ec-5b5a8d65bd8c req-170751cc-28ef-4219-aac6-21acd44e2995 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:41.809950 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-90936226-be47-42a6-89ec-5b5a8d65bd8c req-170751cc-28ef-4219-aac6-21acd44e2995 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:41.810743 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-90936226-be47-42a6-89ec-5b5a8d65bd8c req-170751cc-28ef-4219-aac6-21acd44e2995 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:41.834615 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-90936226-be47-42a6-89ec-5b5a8d65bd8c req-170751cc-28ef-4219-aac6-21acd44e2995 service nova] Creating event network-vif-plugged:4aef1009-d2d3-4c6f-af8a-c5134e19fa37 for instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 on hv-cinder-maste Sep 25 01:02:41.844384 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-90936226-be47-42a6-89ec-5b5a8d65bd8c req-170751cc-28ef-4219-aac6-21acd44e2995 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.064814 Sep 25 01:02:41.845537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 383/779] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:02:41 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 68 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:02:42.410722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "9269fe74-09ef-4975-aa65-dfccb39dcb36"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:42.414195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:42.421225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:42.421750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:42.422195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:42.423071 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:42.423530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:42.423960 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:42.531976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:42.533414 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:42.542703 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:42.543497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:42.544055 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:42.544938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:42.545734 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:42.546268 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:42.707285 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:42.723383 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:42.727076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:42.729350 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-575c370e-e22d-4329-a4d9-b8750275456c tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.204162 Sep 25 01:02:42.730411 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 384/780] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:42 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 207 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:42.759957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestShelveInstance-server-682240723", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestShelveInstance-102314433", "security_groups": [{"name": "tempest-secgroup-smoke-1405492936"}], "networks": [{"uuid": "c62c61a7-64ad-4bd4-8ce7-b21b978664a6"}], "block_device_mapping_v2": [{"uuid": "29f1a0b7-e71c-4dba-bf67-8867381dbb82", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": true}]}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:42.904388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] validate_networks() for [('c62c61a7-64ad-4bd4-8ce7-b21b978664a6', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:02:43.176133 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=91,nova_cell1:INSERT=3,nova_cell1:UPDATE=6 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:43.188082 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cf99f59c-5914-4d91-a273-8d12eabd0d52 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "POST /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.781280 Sep 25 01:02:43.188696 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 397/781] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 01:02:42 2022] POST /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments => generated 194 bytes in 783 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:02:43.523953 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:02:43.524381 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:02:43.524857 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:02:43.525293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:02:43.525708 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:02:43.538455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:43.539714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:43.540456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:43.541784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:43.542576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:43.543249 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:43.566596 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Getting quotas for project 69a3d6ce251e480d866a86ed9f6f2eb9. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:02:43.573595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Getting quotas for user 598ec83f7b7f4a9f90543218852aaf14 and project 69a3d6ce251e480d866a86ed9f6f2eb9. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:02:43.670919 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:02:43.673198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='29f1a0b7-e71c-4dba-bf67-8867381dbb82',volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:02:43.768160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:43.772617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:43.787200 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:43.788716 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:43.789765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:43.791824 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:43.792599 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:43.793676 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:43.965676 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-390aa63b-e609-459b-a9ce-0c0aada1fdb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.211121 Sep 25 01:02:43.967453 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 385/782] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:02:42 2022] POST /compute/v2.1/servers => generated 406 bytes in 1214 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 01:02:43.983903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6bed83e6-e516-4998-ab80-9e71b8ffe1b2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:43.985171 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6bed83e6-e516-4998-ab80-9e71b8ffe1b2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:44.073164 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6bed83e6-e516-4998-ab80-9e71b8ffe1b2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:44.089639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6bed83e6-e516-4998-ab80-9e71b8ffe1b2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1087 microversion: 2.1 time: 0.109612 Sep 25 01:02:44.092642 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 386/783] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:43 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1087 bytes in 114 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:02:44.112895 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 25 01:02:44.113549 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Sun Sep 25 01:02:44 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 (ip 10.222.0.52) !!! Sep 25 01:02:44.144655 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:44.154630 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:44.156475 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:44.157700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ed05fcf4-f958-40cd-a0cd-da5e091182ef tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1731 microversion: 2.20 time: 0.401020 Sep 25 01:02:44.158446 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 398/784] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:43 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1731 bytes in 404 msecs (HTTP/1.1 200) 9 headers in 360 bytes (2 switches on core 0) Sep 25 01:02:45.128337 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:45.129505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:45.141415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:45.142582 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:45.143744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:45.144876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:45.145883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:45.146799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:45.204436 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:45.205990 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:45.220457 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:45.221427 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:45.222846 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:45.224865 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:45.226519 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:45.241699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:45.268255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:45.290259 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f6d154db-5330-4215-bb2b-2b469ec99999 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1185 microversion: 2.1 time: 0.166439 Sep 25 01:02:45.291021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 387/785] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:45 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1185 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:02:45.438335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:45.456359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:45.460612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:45.463072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b3d41223-1775-44ba-b7ae-547ee879ff33 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1718 microversion: 2.20 time: 0.265612 Sep 25 01:02:45.464356 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 399/786] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:02:45 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1718 bytes in 268 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:02:46.319333 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:46.320232 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:46.328948 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:46.329796 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:46.330409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:46.331269 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:46.331914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:46.332436 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:46.404377 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:46.412941 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-696805ac-7d25-4ac5-ad5e-7b09e2aa72e6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1193 microversion: 2.1 time: 0.099860 Sep 25 01:02:46.413471 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 388/787] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:46 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1193 bytes in 101 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:47.436919 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:47.437633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:47.445442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:47.446100 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:47.446578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:47.447372 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:47.447892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:47.448351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:47.539600 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:47.554253 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f443b29b-4264-4608-bdfa-6dc4c5006ef9 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1193 microversion: 2.1 time: 0.121382 Sep 25 01:02:47.555496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 400/788] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:47 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1193 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:48.596390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:48.598727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:48.612263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:48.613525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:48.614616 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:48.617226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:48.618360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:48.619274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:48.733818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:48.748659 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-48d411d1-17fc-4cd5-92d3-85106fee1882 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1203 microversion: 2.1 time: 0.161053 Sep 25 01:02:48.749722 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 389/789] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:48 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1203 bytes in 164 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:49.778384 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:49.779829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:49.787951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:49.788455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:49.788897 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:49.789681 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:49.790563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:49.791217 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:49.942377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:49.953371 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fba95f51-ce0b-43ac-b3fd-6f09837cd477 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1271 microversion: 2.1 time: 0.178801 Sep 25 01:02:49.954306 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 401/790] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:49 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1271 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:50.443565 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-34986835-0e73-4709-b7c2-118c83481343 req-31ba7133-c06e-473f-aaf0-387c7350b414 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb66c484-73f0-441b-a575-f506a0caca8a", "tag": "7589db85-95c8-412c-a8f7-5db156ef7906"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:50.454878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-34986835-0e73-4709-b7c2-118c83481343 req-31ba7133-c06e-473f-aaf0-387c7350b414 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:50.455655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-34986835-0e73-4709-b7c2-118c83481343 req-31ba7133-c06e-473f-aaf0-387c7350b414 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:50.456158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-34986835-0e73-4709-b7c2-118c83481343 req-31ba7133-c06e-473f-aaf0-387c7350b414 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:50.469783 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-34986835-0e73-4709-b7c2-118c83481343 req-31ba7133-c06e-473f-aaf0-387c7350b414 service nova] Creating event network-changed:7589db85-95c8-412c-a8f7-5db156ef7906 for instance eb66c484-73f0-441b-a575-f506a0caca8a on hv-cinder-maste Sep 25 01:02:50.481774 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-34986835-0e73-4709-b7c2-118c83481343 req-31ba7133-c06e-473f-aaf0-387c7350b414 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042319 Sep 25 01:02:50.482809 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 390/791] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:02:50 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:02:50.983535 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:50.984910 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:50.997203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:50.997944 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:50.998542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:50.999690 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:51.000332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:51.001024 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:51.208210 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:51.224441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9ba5b36c-724f-4903-889f-0d0d5872fcfd tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1271 microversion: 2.1 time: 0.247775 Sep 25 01:02:51.225570 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 402/792] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:50 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1271 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:52.258780 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:52.260830 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:52.271471 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:52.272360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:52.273304 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:52.273304 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:52.274500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:52.275384 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:52.275532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:52.276130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:52.291202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:52.292305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:52.293172 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:52.294599 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:52.295571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:52.296404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:52.395734 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lazy-loading 'services' on Instance uuid cb07cc70-0299-47dc-97cb-e3b82c1a47b2 {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:02:52.513812 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-82e4a455-8ee6-4e7b-8474-d699a98ce8aa tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments/9269fe74-09ef-4975-aa65-dfccb39dcb36" status: 202 len: 0 microversion: 2.1 time: 0.265203 Sep 25 01:02:52.514695 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 391/793] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:02:52 2022] DELETE /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments/9269fe74-09ef-4975-aa65-dfccb39dcb36 => generated 0 bytes in 268 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:02:52.529683 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:52.532349 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:52.536853 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:52.539506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:52.540012 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:52.540442 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:52.541395 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:52.541861 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:52.542289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:52.545898 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fb6a0262-ba02-4428-9218-0f08a0cd4455 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.283143 Sep 25 01:02:52.546503 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 403/794] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:52 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 285 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:52.578053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f96ad557-453a-4292-89b6-a714461ea047 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.057812 Sep 25 01:02:52.578648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 392/795] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:02:52 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments => generated 197 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:02:53.570869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:53.571591 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:53.579469 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:53.580246 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:53.580682 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:53.581741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:53.582433 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:53.582876 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:53.595987 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:53.604326 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:53.609838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:53.624559 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:53.625474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:53.626142 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:53.627443 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:53.628280 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:53.629038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:53.678470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-69c542ab-b120-4fef-8fe7-d978ae582fea tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.088722 Sep 25 01:02:53.679545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 393/796] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:02:53 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments => generated 197 bytes in 92 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:02:53.851903 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:53.861623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-03aac542-591b-43f1-844a-7f19d02fad5e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.294903 Sep 25 01:02:53.862381 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 404/797] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:53 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 297 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:02:54.695274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:54.697669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:54.699692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=8 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:54.706348 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:54.706885 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:54.707360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:54.708104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:54.709406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:54.709891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:54.749934 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4e7fb622-40cf-4e3b-b6ae-0d23efbe9102 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.058740 Sep 25 01:02:54.750581 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 394/798] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:02:54 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2/os-volume_attachments => generated 25 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 25 01:02:54.824307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:54.824943 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:54.827587 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=32 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:54.834413 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:54.835150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:54.835596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:54.836402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:54.836864 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:54.837395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:54.868083 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:02:54.908521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:54.909687 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:54.922095 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:54.922974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:54.923760 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:54.924998 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:54.925815 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:54.926560 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:54.966107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-016168f0-47b5-435b-9ecb-11e7cfdac9e8 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 204 len: 0 microversion: 2.1 time: 0.144907 Sep 25 01:02:54.967419 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 405/799] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:02:54 2022] DELETE /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 25 01:02:54.980048 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:54.981433 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:54.995795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:54.995795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:54.995795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:54.995795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:54.996615 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:54.996615 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:55.131882 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:55.141713 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-417496f4-44eb-42d0-a280-1484f5033745 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.242636 Sep 25 01:02:55.142576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 395/800] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:54 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:02:55.165184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:55.173954 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-37bead32-575a-417b-92e1-b0aa500619e3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1681 microversion: 2.1 time: 0.201377 Sep 25 01:02:55.174546 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 406/801] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:54 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1681 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:02:56.175990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:56.176743 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:56.180845 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:56.189148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:56.189786 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:56.190500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:56.191767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:56.193091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:56.193719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:56.197883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=99,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:02:56.202416 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:56.203751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:56.217199 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:56.218130 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:56.218950 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:56.220222 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:56.221000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:56.221793 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:56.388872 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:56.403852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7adc16dd-c4aa-490c-a82a-d7cd751cfb59 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1681 microversion: 2.1 time: 0.208414 Sep 25 01:02:56.404910 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 407/802] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:56 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1681 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:02:56.422184 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:56.432477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a6108587-7a91-4974-a257-d1d0cf7ca772 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.260891 Sep 25 01:02:56.433070 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 396/803] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:56 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:57.442925 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:57.443981 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:57.453426 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:57.454640 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:57.455366 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:57.456567 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:57.457345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:57.458070 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:57.460337 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:57.462527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:57.473069 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:57.474829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:57.475559 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:57.476787 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:57.477628 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:57.478625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:57.692310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:57.702553 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3b7a7f1c-a738-4322-8469-3a6ecef72952 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1681 microversion: 2.1 time: 0.264355 Sep 25 01:02:57.703152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 408/804] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:57 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1681 bytes in 266 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:57.781678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:57.796136 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-91fd66b4-17bd-48ae-b8a7-049fef93347c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.342217 Sep 25 01:02:57.797207 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 397/805] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:57 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 345 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:02:58.165813 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-af8a8428-416a-44a0-a794-82859bd6249b req-de6f3942-04de-4151-b8c5-81c7aed84753 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "cb07cc70-0299-47dc-97cb-e3b82c1a47b2", "name": "network-vif-deleted", "tag": "e90855c6-13bf-47db-8fb7-7226a222a0c7"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:02:58.183511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-af8a8428-416a-44a0-a794-82859bd6249b req-de6f3942-04de-4151-b8c5-81c7aed84753 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:58.184024 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-af8a8428-416a-44a0-a794-82859bd6249b req-de6f3942-04de-4151-b8c5-81c7aed84753 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:58.184554 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-af8a8428-416a-44a0-a794-82859bd6249b req-de6f3942-04de-4151-b8c5-81c7aed84753 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:58.199083 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-af8a8428-416a-44a0-a794-82859bd6249b req-de6f3942-04de-4151-b8c5-81c7aed84753 service nova] Creating event network-vif-deleted:e90855c6-13bf-47db-8fb7-7226a222a0c7 for instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 on hv-cinder-maste Sep 25 01:02:58.205714 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-af8a8428-416a-44a0-a794-82859bd6249b req-de6f3942-04de-4151-b8c5-81c7aed84753 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050066 Sep 25 01:02:58.206351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 409/806] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:02:58 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:02:58.747246 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:58.748502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:58.758024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:58.758953 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:58.760322 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:58.761806 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:58.762674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:58.763615 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:58.812097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:58.812701 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:58.813225 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:58.813794 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lazy-loading 'fault' on Instance uuid cb07cc70-0299-47dc-97cb-e3b82c1a47b2 {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:02:58.825794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:58.827287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:58.835386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:58.836269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:58.836936 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:58.838014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:58.838779 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:58.839214 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:58.930459 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '96386a16f0b54b7c94d32bd948caf118', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f08425c00f0c49ddbe42b21ddbc7ff3f', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:58.948840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-606b2f92-5461-4181-bddc-c4778b1893b0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 200 len: 1331 microversion: 2.1 time: 0.205883 Sep 25 01:02:58.950846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 398/807] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:58 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 1331 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:02:59.034326 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:02:59.048813 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-95b47c9f-69d1-4308-af43-b018e5184e1c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.227750 Sep 25 01:02:59.049899 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 410/808] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:58 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:02:59.988194 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:02:59.988821 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:02:59.995553 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:59.996054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:59.996507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:02:59.997238 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:02:59.997744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:02:59.998179 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:00.014725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 HTTP/1.1 Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:00.016423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:00.018978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'cb07cc70-0299-47dc-97cb-e3b82c1a47b2'} Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:00.020655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:00.023428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:00.025811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:00.025811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bba47455-8ed8-4752-93a6-2346f3bc4ef1 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 404 len: 111 microversion: 2.1 time: 0.034173 Sep 25 01:03:00.025811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 399/809] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:02:59 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:03:00.030141 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6adfbcbf-f8b6-4a3c-bd26-87330ae066d3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '.version_select at 0x7fe22d72cee0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:00.055588 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6adfbcbf-f8b6-4a3c-bd26-87330ae066d3 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-412498492" status: 202 len: 0 microversion: 2.1 time: 0.029489 Sep 25 01:03:00.056308 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 411/810] 10.222.0.52 () {58 vars in 1218 bytes} [Sun Sep 25 01:03:00 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-412498492 => generated 0 bytes in 32 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:03:00.074837 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:00.075632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:00.082142 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:00.082638 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:00.083674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:00.084625 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:00.085100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:00.085729 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:00.415257 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:00.428816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8f21b576-905c-4226-8454-7cad90592d60 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.356626 Sep 25 01:03:00.429852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 400/811] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:00 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 360 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:03:01.454595 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:01.455492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:01.467825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:01.468374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:01.468885 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:01.470228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:01.470712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:01.471149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:01.632506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:01.641327 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b78121f8-220c-46f2-878c-cf70fdda81ee tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.191297 Sep 25 01:03:01.641932 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 412/812] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:01 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:03:01.841153 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-6888cc56-8ad5-44c2-a56d-3cd7f766780c req-72597ac7-7795-4eb9-adaa-5ab635124a4c service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb66c484-73f0-441b-a575-f506a0caca8a", "name": "network-vif-plugged", "status": "completed", "tag": "7589db85-95c8-412c-a8f7-5db156ef7906"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:01.854596 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-6888cc56-8ad5-44c2-a56d-3cd7f766780c req-72597ac7-7795-4eb9-adaa-5ab635124a4c service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:01.854843 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-6888cc56-8ad5-44c2-a56d-3cd7f766780c req-72597ac7-7795-4eb9-adaa-5ab635124a4c service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:01.855257 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-6888cc56-8ad5-44c2-a56d-3cd7f766780c req-72597ac7-7795-4eb9-adaa-5ab635124a4c service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:01.875984 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-6888cc56-8ad5-44c2-a56d-3cd7f766780c req-72597ac7-7795-4eb9-adaa-5ab635124a4c service nova] Creating event network-vif-plugged:7589db85-95c8-412c-a8f7-5db156ef7906 for instance eb66c484-73f0-441b-a575-f506a0caca8a on hv-cinder-maste Sep 25 01:03:01.885877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-6888cc56-8ad5-44c2-a56d-3cd7f766780c req-72597ac7-7795-4eb9-adaa-5ab635124a4c service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.048637 Sep 25 01:03:01.887195 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 401/813] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:03:01 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 51 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:03:02.218401 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:02.219364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:02.227718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:02.228222 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:02.228689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:02.229456 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:02.229929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:02.230362 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:02.247013 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: DELETE /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 HTTP/1.1 Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:02.248754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.251423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': 'cb07cc70-0299-47dc-97cb-e3b82c1a47b2'} Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:02.253002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.255973 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.257496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:02.257496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ce8de1cc-609a-4d87-a6a4-484e1f954a99 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 404 len: 111 microversion: 2.1 time: 0.035825 Sep 25 01:03:02.257496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 413/814] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:03:02 2022] DELETE /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:03:02.260277 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:02.260994 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: cb07cc70-0299-47dc-97cb-e3b82c1a47b2] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:02.268275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:02.268929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:02.269580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:02.270505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:02.271077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:02.271611 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:02.287199 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 HTTP/1.1 Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:02.288985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.292244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'cb07cc70-0299-47dc-97cb-e3b82c1a47b2'} Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:02.294328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. Sep 25 01:03:02.296772 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:02.299161 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance cb07cc70-0299-47dc-97cb-e3b82c1a47b2 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:02.299161 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f9bfd589-4a37-4961-9f40-a522472cdfa6 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2" status: 404 len: 111 microversion: 2.1 time: 0.034627 Sep 25 01:03:02.299161 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 402/815] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:02 2022] GET /compute/v2.1/servers/cb07cc70-0299-47dc-97cb-e3b82c1a47b2 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:03:02.668343 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:02.669077 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:02.676174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:02.678058 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:02.678675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:02.679581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:02.680150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:02.680726 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:02.907256 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:02.916319 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-47ae41da-762d-4a30-8cbf-8b639f53e41e tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1259 microversion: 2.1 time: 0.251793 Sep 25 01:03:02.916910 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 414/816] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:02 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1259 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:03:03.469051 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:03.470775 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:03.482576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.483674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.484676 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.485948 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.487090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.488057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.516510 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 HTTP/1.1 Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:03.519886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.522970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'e53662f0-18c6-40e1-99da-07e0065395e1'} Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:03.524558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.528542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.532181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:03.532181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9e759684-ce98-47a8-93cd-318c23fdfba0 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 404 len: 111 microversion: 2.1 time: 0.060698 Sep 25 01:03:03.533353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 403/817] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:03:03 2022] DELETE /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 111 bytes in 70 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:03:03.539145 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:03.539748 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: e53662f0-18c6-40e1-99da-07e0065395e1] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:03.546598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.547088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.547529 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.548244 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.548700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.549126 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.554170 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:03.565062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 HTTP/1.1 Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:03.566843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.568996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': 'e53662f0-18c6-40e1-99da-07e0065395e1'} Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:03.570174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. Sep 25 01:03:03.571614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.573050 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance e53662f0-18c6-40e1-99da-07e0065395e1 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:03.573050 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3697a1f8-0fab-4e5a-97fd-0a364bae8cc5 tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1" status: 404 len: 111 microversion: 2.1 time: 0.034806 Sep 25 01:03:03.573050 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 415/818] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:03 2022] GET /compute/v2.1/servers/e53662f0-18c6-40e1-99da-07e0065395e1 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:03:03.579166 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:03.579664 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:03.587053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.587544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.587975 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.588692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.589236 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.589730 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.607907 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 HTTP/1.1 Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:03.610525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.612134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '6892e391-59db-43ba-8843-ce34ade53390'} Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:03.613083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.614923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.616458 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:03.616458 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0145e7e9-908d-41e5-94bf-00ab3661035e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 404 len: 111 microversion: 2.1 time: 0.037496 Sep 25 01:03:03.616458 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 404/819] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:03:03 2022] DELETE /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:03:03.622955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:03.623574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: 6892e391-59db-43ba-8843-ce34ade53390] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:03.631740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.632233 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.632672 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.633406 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.633870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.634274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.650673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 HTTP/1.1 Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:03.652335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.654606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '6892e391-59db-43ba-8843-ce34ade53390'} Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:03.656012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. Sep 25 01:03:03.659177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:03.661977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance 6892e391-59db-43ba-8843-ce34ade53390 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:03.661977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-07595a23-f396-4d84-a858-c4ae32b743ab tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390" status: 404 len: 111 microversion: 2.1 time: 0.034819 Sep 25 01:03:03.661977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 416/820] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:03 2022] GET /compute/v2.1/servers/6892e391-59db-43ba-8843-ce34ade53390 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:03:03.940909 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:03.941477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:03.948125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.948610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.949047 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:03.949782 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:03.950230 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:03.950654 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:04.109927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:04.118883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-cacee9df-82cc-4130-b3b2-fb59bb4c2996 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1443 microversion: 2.1 time: 0.182451 Sep 25 01:03:04.119465 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 405/821] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:03 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1443 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:03:04.142078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:04.143470 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:04.150542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:04.151046 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:04.151479 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:04.152191 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:04.152723 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:04.153150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:04.306416 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:04.315345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-479bbe22-cc88-4a53-abaa-3b35e9e6ddad tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1443 microversion: 2.1 time: 0.177622 Sep 25 01:03:04.315918 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 417/822] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:04 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1443 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:03:04.825427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:04.826133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:04.833879 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:04.834441 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:04.834878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:04.835597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:04.836048 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:04.836466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:04.853380 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc HTTP/1.1 Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:04.855045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.857803 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'af139b53-2db6-43ea-9e69-27dfbb9304bc'} Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:04.859091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.861876 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.865039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:04.865039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-784463b3-7968-41db-bfc4-9a2605b12c4a tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "DELETE /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 404 len: 111 microversion: 2.1 time: 0.035926 Sep 25 01:03:04.865039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 406/823] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:03:04 2022] DELETE /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:03:04.866803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:04.867474 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] [instance: af139b53-2db6-43ea-9e69-27dfbb9304bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:04.874253 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:04.874744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:04.875181 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:04.876060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:04.876526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:04.877112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:04.892808 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] HTTP exception thrown: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Request method failure captured: Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc HTTP/1.1 Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cXZ_Sl55H566Pw29-X9PXTVItS24WO5sPYlFCTVfnaAYc7zDFM-lFGFZjYYfzrM2Mw-gszI97lr8PhAoiw3ShF4QecK2PZBM-9vFry4ODLQ1YdVUQ-Edqt8IZXWWjtvcMfrO73kTc5qZ5sLH9FlxANjByWFoID4Pvkgh3CnJ8YtAY Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:04.894447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f08425c00f0c49ddbe42b21ddbc7ff3f", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f08425c00f0c49ddbe42b21ddbc7ff3f"}]}] Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f08425c00f0c49ddbe42b21ddbc7ff3f Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeNegativeTest-381704141 Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 96386a16f0b54b7c94d32bd948caf118 Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeNegativeTest-381704141-project Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.897018 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': 'af139b53-2db6-43ea-9e69-27dfbb9304bc'} Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:04.898486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. Sep 25 01:03:04.900857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:04.903055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] Returning 404 to user: Instance af139b53-2db6-43ea-9e69-27dfbb9304bc could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:04.903055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f6a5c541-23a7-42f2-b3fe-20f28e926c4e tempest-AttachVolumeNegativeTest-381704141 tempest-AttachVolumeNegativeTest-381704141-project] 10.222.0.52 "GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc" status: 404 len: 111 microversion: 2.1 time: 0.034341 Sep 25 01:03:04.903055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 418/824] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:04 2022] GET /compute/v2.1/servers/af139b53-2db6-43ea-9e69-27dfbb9304bc => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:03:07.186687 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-708f9a16-422a-4e46-b516-a5be83c386b1 req-2a17b35b-d8ea-4630-8355-6ed5017a0165 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb66c484-73f0-441b-a575-f506a0caca8a", "tag": "7589db85-95c8-412c-a8f7-5db156ef7906"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:07.200926 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-708f9a16-422a-4e46-b516-a5be83c386b1 req-2a17b35b-d8ea-4630-8355-6ed5017a0165 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:07.201764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-708f9a16-422a-4e46-b516-a5be83c386b1 req-2a17b35b-d8ea-4630-8355-6ed5017a0165 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:07.202438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-708f9a16-422a-4e46-b516-a5be83c386b1 req-2a17b35b-d8ea-4630-8355-6ed5017a0165 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:07.219354 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-708f9a16-422a-4e46-b516-a5be83c386b1 req-2a17b35b-d8ea-4630-8355-6ed5017a0165 service nova] Creating event network-changed:7589db85-95c8-412c-a8f7-5db156ef7906 for instance eb66c484-73f0-441b-a575-f506a0caca8a on hv-cinder-maste Sep 25 01:03:07.234977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-708f9a16-422a-4e46-b516-a5be83c386b1 req-2a17b35b-d8ea-4630-8355-6ed5017a0165 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.047439 Sep 25 01:03:07.241136 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 407/825] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:03:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:03:11.229091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-abcace67-b386-4ece-9dec-29e2cd4150cc tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Action: 'create', calling method: .version_select at 0x7fe22d6d0af0>, body: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1034194638", "metadata": {"Type": "work"}}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-abcace67-b386-4ece-9dec-29e2cd4150cc tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Request method failure captured: Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 122 Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6jO738lmLMHHzwlzWCMui2uDzJgaK3wV03cJz4OG2suQunniUlIpdAwiZTojzamkme7igwxj7jeu-wqqfXXEGbTaa7GIpVGvptoQx29fq_nthk4rvEAjvpTndPp2tMf_aOilPq0tsi9iovKWIQhmF5BuSCGZSVGOzZCNkDmd5BpGGM Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:11.244394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_3b39228a17314fa49ffb00c024e7dc68", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}] Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 43e16e8590f445348c9b3d121f66ecb9 Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {"volume": {"size": "#$%", "display_name": "tempest-VolumesNegativeTest-Volume-1034194638", "metadata": {"Type": "work"}}} Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: Sep 25 01:03:11.246877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'body': {'volume': {'size': '#$%', 'display_name': 'tempest-VolumesNegativeTest-Volume-1034194638', 'metadata': {'Type': 'work'}}}} Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise error Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '#$%' does not match '^[0-9]+$' Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi '#$%' Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:11.248275 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 25 01:03:11.250524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 25 01:03:11.253131 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' Sep 25 01:03:11.253131 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.253131 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-abcace67-b386-4ece-9dec-29e2cd4150cc tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Returning 400 to user: Invalid input for field/attribute size. Value: #$%. '#$%' does not match '^[0-9]+$' {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:11.253131 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-abcace67-b386-4ece-9dec-29e2cd4150cc tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 400 len: 127 microversion: 2.1 time: 0.188937 Sep 25 01:03:11.253131 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 419/826] 10.222.0.52 () {60 vars in 1156 bytes} [Sun Sep 25 01:03:11 2022] POST /compute/v2.1/os-volumes => generated 127 bytes in 191 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 25 01:03:11.260752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-02ac33d4-7fce-48aa-895e-e93b53a0463e tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Action: 'create', calling method: .version_select at 0x7fe22ee04dc0>, body: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-284603852", "metadata": {"Type": "work"}}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-02ac33d4-7fce-48aa-895e-e93b53a0463e tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Request method failure captured: Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 119 Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6jO738lmLMHHzwlzWCMui2uDzJgaK3wV03cJz4OG2suQunniUlIpdAwiZTojzamkme7igwxj7jeu-wqqfXXEGbTaa7GIpVGvptoQx29fq_nthk4rvEAjvpTndPp2tMf_aOilPq0tsi9iovKWIQhmF5BuSCGZSVGOzZCNkDmd5BpGGM Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:11.265715 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_3b39228a17314fa49ffb00c024e7dc68", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}] Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 43e16e8590f445348c9b3d121f66ecb9 Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {"volume": {"size": "0", "display_name": "tempest-VolumesNegativeTest-Volume-284603852", "metadata": {"Type": "work"}}} Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: Sep 25 01:03:11.268197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'body': {'volume': {'size': '0', 'display_name': 'tempest-VolumesNegativeTest-Volume-284603852', 'metadata': {'Type': 'work'}}}} Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise error Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: 0 is less than the minimum of 1 Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Failed validating 'minimum' in schema['properties']['volume']['properties']['size']: Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi '0' Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:11.269811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 25 01:03:11.275262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 25 01:03:11.277524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 Sep 25 01:03:11.277524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.277524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-02ac33d4-7fce-48aa-895e-e93b53a0463e tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Returning 400 to user: Invalid input for field/attribute size. Value: 0. 0 is less than the minimum of 1 {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:11.277524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-02ac33d4-7fce-48aa-895e-e93b53a0463e tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 400 len: 125 microversion: 2.1 time: 0.011741 Sep 25 01:03:11.277524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 408/827] 10.222.0.52 () {60 vars in 1156 bytes} [Sun Sep 25 01:03:11 2022] POST /compute/v2.1/os-volumes => generated 125 bytes in 14 msecs (HTTP/1.1 400) 9 headers in 381 bytes (2 switches on core 0) Sep 25 01:03:11.278772 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4182a3f5-9876-40b0-aa0a-c5e8675a04ba tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Action: 'create', calling method: .version_select at 0x7fe22d72c700>, body: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-99868620", "metadata": {"Type": "work"}}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4182a3f5-9876-40b0-aa0a-c5e8675a04ba tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Request method failure captured: Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: POST /compute/v2.1/os-volumes HTTP/1.1 Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 117 Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6jO738lmLMHHzwlzWCMui2uDzJgaK3wV03cJz4OG2suQunniUlIpdAwiZTojzamkme7igwxj7jeu-wqqfXXEGbTaa7GIpVGvptoQx29fq_nthk4rvEAjvpTndPp2tMf_aOilPq0tsi9iovKWIQhmF5BuSCGZSVGOzZCNkDmd5BpGGM Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:11.283759 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_3b39228a17314fa49ffb00c024e7dc68", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}] Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 43e16e8590f445348c9b3d121f66ecb9 Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {"volume": {"size": "", "display_name": "tempest-VolumesNegativeTest-Volume-99868620", "metadata": {"Type": "work"}}} Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: Sep 25 01:03:11.287182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'body': {'volume': {'size': '', 'display_name': 'tempest-VolumesNegativeTest-Volume-99868620', 'metadata': {'Type': 'work'}}}} Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 302, in validate Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi self.validator.validate(*args, **kwargs) Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/jsonschema/validators.py", line 303, in validate Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise error Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi jsonschema.exceptions.ValidationError: '' does not match '^[0-9]+$' Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Failed validating 'pattern' in schema['properties']['volume']['properties']['size']: Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi {'minimum': 1, 'pattern': '^[0-9]+$', 'type': ['integer', 'string']} Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi On instance['volume']: Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi '' Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:11.289386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 107, in wrapper Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi _schema_validation_helper(request_body_schema, kwargs['body'], Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/__init__.py", line 88, in _schema_validation_helper Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi schema_validator.validate(target) Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/validation/validators.py", line 331, in validate Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.ValidationError(detail=detail) Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.ValidationError: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 359, in __exit__ Sep 25 01:03:11.292269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(exception.ConvertedException( Sep 25 01:03:11.294770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' Sep 25 01:03:11.294770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.294770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-4182a3f5-9876-40b0-aa0a-c5e8675a04ba tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Returning 400 to user: Invalid input for field/attribute size. Value: . '' does not match '^[0-9]+$' {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:11.294770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-4182a3f5-9876-40b0-aa0a-c5e8675a04ba tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] 10.222.0.52 "POST /compute/v2.1/os-volumes" status: 400 len: 121 microversion: 2.1 time: 0.010552 Sep 25 01:03:11.294770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 420/828] 10.222.0.52 () {60 vars in 1156 bytes} [Sun Sep 25 01:03:11 2022] POST /compute/v2.1/os-volumes => generated 121 bytes in 12 msecs (HTTP/1.1 400) 9 headers in 381 bytes (1 switches on core 0) Sep 25 01:03:11.300131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ad044575-44f4-4f62-900f-19830cb57dde tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Calling method '.version_select at 0x7fe22d796e50>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:11.342263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-ad044575-44f4-4f62-900f-19830cb57dde tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] HTTP exception thrown: Volume tempest-invalid-600408753 could not be found. Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ad044575-44f4-4f62-900f-19830cb57dde tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Request method failure captured: Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/os-volumes/tempest-invalid-600408753 HTTP/1.1 Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6jO738lmLMHHzwlzWCMui2uDzJgaK3wV03cJz4OG2suQunniUlIpdAwiZTojzamkme7igwxj7jeu-wqqfXXEGbTaa7GIpVGvptoQx29fq_nthk4rvEAjvpTndPp2tMf_aOilPq0tsi9iovKWIQhmF5BuSCGZSVGOzZCNkDmd5BpGGM Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:11.346250 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_3b39228a17314fa49ffb00c024e7dc68", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}] Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 43e16e8590f445348c9b3d121f66ecb9 Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Volume tempest-invalid-600408753 could not be found. Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'tempest-invalid-600408753'} Sep 25 01:03:11.349104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume tempest-invalid-600408753 could not be found. (HTTP 404) (Request-ID: req-c8908681-7f47-4ef9-9241-1f47b37b0549) Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.350662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 25 01:03:11.353191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume tempest-invalid-600408753 could not be found. Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:11.355927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume tempest-invalid-600408753 could not be found. Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume tempest-invalid-600408753 could not be found. Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ad044575-44f4-4f62-900f-19830cb57dde tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Returning 404 to user: Volume tempest-invalid-600408753 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ad044575-44f4-4f62-900f-19830cb57dde tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/tempest-invalid-600408753" status: 404 len: 98 microversion: 2.1 time: 0.056813 Sep 25 01:03:11.358580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 409/829] 10.222.0.52 () {58 vars in 1215 bytes} [Sun Sep 25 01:03:11 2022] DELETE /compute/v2.1/os-volumes/tempest-invalid-600408753 => generated 98 bytes in 59 msecs (HTTP/1.1 404) 9 headers in 378 bytes (2 switches on core 0) Sep 25 01:03:11.367929 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-095d3f1f-d32a-488d-8b02-144cd094a4e7 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004807 Sep 25 01:03:11.368476 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 421/830] 10.222.0.52 () {58 vars in 1140 bytes} [Sun Sep 25 01:03:11 2022] DELETE /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (2 switches on core 0) Sep 25 01:03:11.379118 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4a6cb36a-4344-4cd8-87a1-1a11753f202e tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/" status: 404 len: 112 microversion: - time: 0.004157 Sep 25 01:03:11.379807 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 410/831] 10.222.0.52 () {58 vars in 1137 bytes} [Sun Sep 25 01:03:11 2022] GET /compute/v2.1/os-volumes/ => generated 112 bytes in 6 msecs (HTTP/1.1 404) 5 headers in 228 bytes (1 switches on core 0) Sep 25 01:03:11.392574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-515c1a7a-843a-453c-b421-70d195e72ef0 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Calling method '.version_select at 0x7fe22d7829d0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:11.431586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-515c1a7a-843a-453c-b421-70d195e72ef0 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] HTTP exception thrown: Volume abe7b846-cd1e-4bde-af3f-e8f996e0924a could not be found. Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-515c1a7a-843a-453c-b421-70d195e72ef0 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Request method failure captured: Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: DELETE /compute/v2.1/os-volumes/abe7b846-cd1e-4bde-af3f-e8f996e0924a HTTP/1.1 Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6jO738lmLMHHzwlzWCMui2uDzJgaK3wV03cJz4OG2suQunniUlIpdAwiZTojzamkme7igwxj7jeu-wqqfXXEGbTaa7GIpVGvptoQx29fq_nthk4rvEAjvpTndPp2tMf_aOilPq0tsi9iovKWIQhmF5BuSCGZSVGOzZCNkDmd5BpGGM Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:11.433647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_3b39228a17314fa49ffb00c024e7dc68", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}] Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 43e16e8590f445348c9b3d121f66ecb9 Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Volume abe7b846-cd1e-4bde-af3f-e8f996e0924a could not be found. Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': 'abe7b846-cd1e-4bde-af3f-e8f996e0924a'} Sep 25 01:03:11.435951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume abe7b846-cd1e-4bde-af3f-e8f996e0924a could not be found. (HTTP 404) (Request-ID: req-a6bbed41-d9f1-4807-ba9c-f89b6f32123d) Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.437700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 136, in delete Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi self.volume_api.delete(context, id) Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 676, in delete Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi cinderclient(context).volumes.delete(volume_id) Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 235, in delete Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self._delete(loc) Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 313, in _delete Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.delete(url) Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 233, in delete Sep 25 01:03:11.440215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'DELETE', **kwargs) Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume abe7b846-cd1e-4bde-af3f-e8f996e0924a could not be found. Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 140, in delete Sep 25 01:03:11.443571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume abe7b846-cd1e-4bde-af3f-e8f996e0924a could not be found. Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume abe7b846-cd1e-4bde-af3f-e8f996e0924a could not be found. Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-515c1a7a-843a-453c-b421-70d195e72ef0 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Returning 404 to user: Volume abe7b846-cd1e-4bde-af3f-e8f996e0924a could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-515c1a7a-843a-453c-b421-70d195e72ef0 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] 10.222.0.52 "DELETE /compute/v2.1/os-volumes/abe7b846-cd1e-4bde-af3f-e8f996e0924a" status: 404 len: 109 microversion: 2.1 time: 0.049311 Sep 25 01:03:11.445613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 422/832] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 01:03:11 2022] DELETE /compute/v2.1/os-volumes/abe7b846-cd1e-4bde-af3f-e8f996e0924a => generated 109 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:03:11.451695 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ef169aec-3200-4620-80b0-28c287425459 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Calling method '.version_select at 0x7fe22dcf7310>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:11.489890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-ef169aec-3200-4620-80b0-28c287425459 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] HTTP exception thrown: Volume 1d209695-92ba-4d96-9a7a-450b8c01767c could not be found. Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ef169aec-3200-4620-80b0-28c287425459 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Request method failure captured: Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/os-volumes/1d209695-92ba-4d96-9a7a-450b8c01767c HTTP/1.1 Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6jO738lmLMHHzwlzWCMui2uDzJgaK3wV03cJz4OG2suQunniUlIpdAwiZTojzamkme7igwxj7jeu-wqqfXXEGbTaa7GIpVGvptoQx29fq_nthk4rvEAjvpTndPp2tMf_aOilPq0tsi9iovKWIQhmF5BuSCGZSVGOzZCNkDmd5BpGGM Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:11.492058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_3b39228a17314fa49ffb00c024e7dc68", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/3b39228a17314fa49ffb00c024e7dc68"}]}] Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 3b39228a17314fa49ffb00c024e7dc68 Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesNegativeTest-1982944763 Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 43e16e8590f445348c9b3d121f66ecb9 Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesNegativeTest-1982944763-project Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Volume 1d209695-92ba-4d96-9a7a-450b8c01767c could not be found. Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '1d209695-92ba-4d96-9a7a-450b8c01767c'} Sep 25 01:03:11.494677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi cinderclient.exceptions.NotFound: Volume 1d209695-92ba-4d96-9a7a-450b8c01767c could not be found. (HTTP 404) (Request-ID: req-92362927-0889-4cbb-9f66-3144b9258e74) Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.496448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 120, in show Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi vol = self.volume_api.get(context, id) Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 395, in wrapper Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, *args, **kwargs) Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 435, in wrapper Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi _reraise(exception.VolumeNotFound(volume_id=volume_id)) Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 486, in _reraise Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise desired_exc.with_traceback(sys.exc_info()[2]) Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 433, in wrapper Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi res = method(self, ctx, volume_id, *args, **kwargs) Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/volume/cinder.py", line 502, in get Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi item = cinderclient( Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/v3/volumes_base.py", line 202, in get Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._get("/volumes/%s" % volume_id, "volume") Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/base.py", line 297, in _get Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi resp, body = self.api.client.get(url) Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 224, in get Sep 25 01:03:11.499031 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self._cs_request(url, 'GET', **kwargs) Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 215, in _cs_request Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return self.request(url, method, **kwargs) Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/usr/local/lib/python3.8/dist-packages/cinderclient/client.py", line 201, in request Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exceptions.from_response(resp, body) Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.VolumeNotFound: Volume 1d209695-92ba-4d96-9a7a-450b8c01767c could not be found. Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 794, in version_select Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return func.func(self, *args, **kwargs) Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/volumes.py", line 122, in show Sep 25 01:03:11.501464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Volume 1d209695-92ba-4d96-9a7a-450b8c01767c could not be found. Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Volume 1d209695-92ba-4d96-9a7a-450b8c01767c could not be found. Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ef169aec-3200-4620-80b0-28c287425459 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] Returning 404 to user: Volume 1d209695-92ba-4d96-9a7a-450b8c01767c could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ef169aec-3200-4620-80b0-28c287425459 tempest-VolumesNegativeTest-1982944763 tempest-VolumesNegativeTest-1982944763-project] 10.222.0.52 "GET /compute/v2.1/os-volumes/1d209695-92ba-4d96-9a7a-450b8c01767c" status: 404 len: 109 microversion: 2.1 time: 0.049209 Sep 25 01:03:11.503762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 411/833] 10.222.0.52 () {58 vars in 1245 bytes} [Sun Sep 25 01:03:11 2022] GET /compute/v2.1/os-volumes/1d209695-92ba-4d96-9a7a-450b8c01767c => generated 109 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:03:24.384677 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=33,nova_api:UPDATE=1,nova_api:DELETE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:24.385693 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=10 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:24.392557 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:24.395188 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:24.406258 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:24.406836 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:24.407373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:24.408169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:24.408689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:24.409230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:24.451453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9ba9a7bb-6995-456c-a88d-8099a8b5f01b tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/os-volume_attachments" status: 200 len: 25 microversion: 2.20 time: 0.067952 Sep 25 01:03:24.452169 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 423/834] 10.222.0.52 () {60 vars in 1343 bytes} [Sun Sep 25 01:03:24 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4/os-volume_attachments => generated 25 bytes in 70 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:03:24.583454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=25 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:24.584315 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=45 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:24.588479 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:24.589092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:24.597056 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:24.597633 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:24.598095 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:24.598819 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:24.599490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:24.599927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:24.629801 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:03:24.740677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ec3d316b-191e-432d-8388-1ec1a1f19337 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "DELETE /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 204 len: 0 microversion: 2.20 time: 0.159151 Sep 25 01:03:24.741813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 412/835] 10.222.0.52 () {60 vars in 1280 bytes} [Sun Sep 25 01:03:24 2022] DELETE /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 25 01:03:24.752230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:24.752975 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:24.759698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:24.760513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:24.760949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:24.761698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:24.762152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:24.762575 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:24.957237 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:24.972980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:24.976068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:24.977819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-71cb1ddd-de02-4b95-ab04-ff90a2067d49 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.229856 Sep 25 01:03:24.978424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 424/836] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:03:24 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:03:26.012655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:26.013552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:26.023311 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:26.024075 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:26.024709 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:26.025642 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:26.026182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:26.026728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:26.231045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:26.245152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:26.246808 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:26.248231 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5b106dc5-8888-482d-82c1-145179f4ea23 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.240854 Sep 25 01:03:26.249003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 413/837] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:03:26 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 243 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:03:27.278517 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:27.280043 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:27.289813 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:27.290403 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:27.290943 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:27.291737 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:27.292448 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:27.293126 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:27.520553 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:27.536581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:27.538903 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:27.540385 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-894b0919-1f1a-45a8-bb03-167e71c2f103 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1724 microversion: 2.20 time: 0.268347 Sep 25 01:03:27.541655 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 425/838] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:03:27 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1724 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:03:28.212189 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-a413b2be-83ca-4b2b-b646-6db3774dc4b3 req-13900a06-ad43-489d-8d10-aea8c93ec4c3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4", "name": "network-vif-deleted", "tag": "4aef1009-d2d3-4c6f-af8a-c5134e19fa37"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:28.230036 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-a413b2be-83ca-4b2b-b646-6db3774dc4b3 req-13900a06-ad43-489d-8d10-aea8c93ec4c3 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:28.230528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-a413b2be-83ca-4b2b-b646-6db3774dc4b3 req-13900a06-ad43-489d-8d10-aea8c93ec4c3 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:28.230965 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-a413b2be-83ca-4b2b-b646-6db3774dc4b3 req-13900a06-ad43-489d-8d10-aea8c93ec4c3 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:28.247201 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-a413b2be-83ca-4b2b-b646-6db3774dc4b3 req-13900a06-ad43-489d-8d10-aea8c93ec4c3 service nova] Creating event network-vif-deleted:4aef1009-d2d3-4c6f-af8a-c5134e19fa37 for instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 on hv-cinder-maste Sep 25 01:03:28.256467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-a413b2be-83ca-4b2b-b646-6db3774dc4b3 req-13900a06-ad43-489d-8d10-aea8c93ec4c3 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052077 Sep 25 01:03:28.257116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 414/839] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:03:28 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:03:28.572510 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:28.573699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:28.586374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:28.587247 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:28.588048 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:28.589375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:28.590205 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:28.590986 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:28.646655 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:28.647104 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:28.647460 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:28.647928 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lazy-loading 'fault' on Instance uuid c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:03:28.698126 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:28.708179 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:28.709991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Policy check for os_compute_api:servers:show:host_status:unknown-only failed with credentials {'is_admin': False, 'user_id': 'dbe551f8c10c4111b4ad7380a6482564', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f09655506e354c9d991f4df9a92f4086', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:28.712068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a6bf7436-6e70-422c-a878-46b4d70e4074 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 200 len: 1372 microversion: 2.20 time: 0.147498 Sep 25 01:03:28.712833 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 426/840] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:03:28 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 1372 bytes in 150 msecs (HTTP/1.1 200) 9 headers in 360 bytes (1 switches on core 0) Sep 25 01:03:29.760157 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:29.761526 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:29.773580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:29.774284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:29.774904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:29.775712 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:29.776260 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:29.776749 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:29.796215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] HTTP exception thrown: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Request method failure captured: Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 HTTP/1.1 Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cgaG8JioxfKBR838yc2ImiVIZR2nzrQQb1CDqfoLEgRRIeq_T_1Jh4FQMpH3FQxKYM_tzvdc2MiB8ugmOEJ2-WAHkXrJ5-Zi0kqn7d2PpMRaT2dcf155Id7yyuHxFCRYc8q5ds3uX1ckQuFFhXY9EnUqrkQnQuqtXoLPq4JYZ5D78 Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Openstack-Nova-Api-Version: 2.20 Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:29.799090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f09655506e354c9d991f4df9a92f4086", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}] Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: dbe551f8c10c4111b4ad7380a6482564 Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:29.802888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4'} Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:29.810537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:29.813363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:29.816077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Returning 404 to user: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:29.816077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0d082bc5-6ecd-49ad-9667-9f135fa10117 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 404 len: 111 microversion: 2.20 time: 0.048561 Sep 25 01:03:29.816077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 415/841] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:03:29 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 111 bytes in 51 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 25 01:03:29.825806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1b6b6be9-9d78-4fd9-abd6-339352ebac3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '.version_select at 0x7fe22d7041f0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:29.862186 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1b6b6be9-9d78-4fd9-abd6-339352ebac3e tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-2072723538" status: 204 len: 0 microversion: 2.20 time: 0.043419 Sep 25 01:03:29.863837 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 427/842] 10.222.0.52 () {60 vars in 1262 bytes} [Sun Sep 25 01:03:29 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-2072723538 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 9 headers in 365 bytes (1 switches on core 0) Sep 25 01:03:31.929767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:31.930603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:31.942227 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:31.942828 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:31.943423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:31.944255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:31.944904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:31.945530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:31.964957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] HTTP exception thrown: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Request method failure captured: Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 HTTP/1.1 Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cgaG8JioxfKBR838yc2ImiVIZR2nzrQQb1CDqfoLEgRRIeq_T_1Jh4FQMpH3FQxKYM_tzvdc2MiB8ugmOEJ2-WAHkXrJ5-Zi0kqn7d2PpMRaT2dcf155Id7yyuHxFCRYc8q5ds3uX1ckQuFFhXY9EnUqrkQnQuqtXoLPq4JYZ5D78 Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Openstack-Nova-Api-Version: 2.20 Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:31.966662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f09655506e354c9d991f4df9a92f4086", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}] Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: dbe551f8c10c4111b4ad7380a6482564 Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:31.974939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4'} Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:31.976721 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:31.979406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:31.982082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Returning 404 to user: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:31.982082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-17a56a15-478a-4af7-b367-65445e76d769 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "DELETE /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 404 len: 111 microversion: 2.20 time: 0.046885 Sep 25 01:03:31.982082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 416/843] 10.222.0.52 () {60 vars in 1280 bytes} [Sun Sep 25 01:03:31 2022] DELETE /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 111 bytes in 48 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 25 01:03:31.983558 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:31.984662 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:31.993555 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:31.994418 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:31.995016 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:31.995947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:31.996646 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:31.997232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:32.016346 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] HTTP exception thrown: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Request method failure captured: Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 HTTP/1.1 Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cgaG8JioxfKBR838yc2ImiVIZR2nzrQQb1CDqfoLEgRRIeq_T_1Jh4FQMpH3FQxKYM_tzvdc2MiB8ugmOEJ2-WAHkXrJ5-Zi0kqn7d2PpMRaT2dcf155Id7yyuHxFCRYc8q5ds3uX1ckQuFFhXY9EnUqrkQnQuqtXoLPq4JYZ5D78 Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Openstack-Nova-Api-Version: 2.20 Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:32.018228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f09655506e354c9d991f4df9a92f4086", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}] Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: dbe551f8c10c4111b4ad7380a6482564 Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:32.026718 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': 'c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4'} Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:32.028395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. Sep 25 01:03:32.030849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:32.034149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Returning 404 to user: Instance c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:32.034149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d518fcff-3502-45fe-8b87-1508eb92af17 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4" status: 404 len: 111 microversion: 2.20 time: 0.045033 Sep 25 01:03:32.034149 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 428/844] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:03:31 2022] GET /compute/v2.1/servers/c09925bc-c9c6-4bd8-9cb1-af5bfdc55af4 => generated 111 bytes in 47 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 25 01:03:33.288077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:33.289042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:33.297912 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:33.298711 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:33.299513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:33.302143 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:33.302737 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:33.303316 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:33.323595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] HTTP exception thrown: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Request method failure captured: Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 HTTP/1.1 Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6cgaG8JioxfKBR838yc2ImiVIZR2nzrQQb1CDqfoLEgRRIeq_T_1Jh4FQMpH3FQxKYM_tzvdc2MiB8ugmOEJ2-WAHkXrJ5-Zi0kqn7d2PpMRaT2dcf155Id7yyuHxFCRYc8q5ds3uX1ckQuFFhXY9EnUqrkQnQuqtXoLPq4JYZ5D78 Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Openstack-Nova-Api-Version: 2.20 Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:03:33.325410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f09655506e354c9d991f4df9a92f4086", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}] Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: dbe551f8c10c4111b4ad7380a6482564 Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.328417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9'} Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:33.329897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.332432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.335214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Returning 404 to user: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:33.335214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a0757c12-545b-4dc8-91cd-68c3de0ab6dd tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "DELETE /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 404 len: 111 microversion: 2.20 time: 0.044766 Sep 25 01:03:33.335214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 417/845] 10.222.0.52 () {60 vars in 1280 bytes} [Sun Sep 25 01:03:33 2022] DELETE /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 111 bytes in 46 msecs (HTTP/1.1 404) 9 headers in 381 bytes (2 switches on core 0) Sep 25 01:03:33.339631 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:33.340478 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] [instance: 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:33.349625 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:33.350167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:33.350613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:33.351353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:33.351814 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:33.352243 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:33.369424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] HTTP exception thrown: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Request method failure captured: Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 HTTP/1.1 Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6cgaG8JioxfKBR838yc2ImiVIZR2nzrQQb1CDqfoLEgRRIeq_T_1Jh4FQMpH3FQxKYM_tzvdc2MiB8ugmOEJ2-WAHkXrJ5-Zi0kqn7d2PpMRaT2dcf155Id7yyuHxFCRYc8q5ds3uX1ckQuFFhXY9EnUqrkQnQuqtXoLPq4JYZ5D78 Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Openstack-Nova-Api-Version: 2.20 Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:03:33.371035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f09655506e354c9d991f4df9a92f4086", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f09655506e354c9d991f4df9a92f4086"}]}] Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f09655506e354c9d991f4df9a92f4086 Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeShelveTestJSON-14329766 Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: dbe551f8c10c4111b4ad7380a6482564 Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeShelveTestJSON-14329766-project Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.373829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9'} Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:03:33.375480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. Sep 25 01:03:33.378376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:03:33.381203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] Returning 404 to user: Instance 4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:03:33.381203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d6118bef-d29d-4171-a9b0-5507c3f7cb71 tempest-AttachVolumeShelveTestJSON-14329766 tempest-AttachVolumeShelveTestJSON-14329766-project] 10.222.0.52 "GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9" status: 404 len: 111 microversion: 2.20 time: 0.037946 Sep 25 01:03:33.381203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 429/846] 10.222.0.52 () {60 vars in 1277 bytes} [Sun Sep 25 01:03:33 2022] GET /compute/v2.1/servers/4eb3d7d6-970a-4c8d-bd97-9982ff56f3b9 => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 381 bytes (1 switches on core 0) Sep 25 01:03:47.192519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=17,nova_api:UPDATE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:47.194011 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=28,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:47.653138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-12f892b8-196a-464d-8714-c8e9e861a222 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'create', calling method: .version_select at 0x7fe22d74a4c0>, body: {"keypair": {"name": "tempest-keypair-31100642", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBKky2yvVffePrDOq4FYU7U2lcSOyD5Vxbsq0TgZA8EHnS4kXXYEOf+MclAWXHZk8tfphZ+bPsh3AwyhGJNj1EzyZVHGdOpbnYBbmnCQ10fsWHA+ivyM52YkXH8cblevA1w=="}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:47.686613 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-12f892b8-196a-464d-8714-c8e9e861a222 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Getting quotas for project f37b53a21b19482b8dcac6c38505a50b. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:03:47.693587 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-12f892b8-196a-464d-8714-c8e9e861a222 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Getting quotas for user 1e515c9c8dcf4a1aa7dd74fb7655adf4 and project f37b53a21b19482b8dcac6c38505a50b. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:03:47.740353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-12f892b8-196a-464d-8714-c8e9e861a222 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 384 microversion: 2.1 time: 0.553404 Sep 25 01:03:47.741816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 418/847] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 01:03:47 2022] POST /compute/v2.1/os-keypairs => generated 384 bytes in 556 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:03:50.162444 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=17,nova_api:DELETE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:50.164596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=30 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:03:50.170960 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1541330529", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "dbea2e08-a361-48c8-95fc-03b3fc326d51"}], "security_groups": [{"name": "tempest-securitygroup--1478945763"}], "key_name": "tempest-keypair-31100642", "user_data": "}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:50.413349 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] validate_networks() for [('dbea2e08-a361-48c8-95fc-03b3fc326d51', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:03:51.372521 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:03:51.373533 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:03:51.373698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:03:51.374113 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:03:51.374564 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:03:51.394722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:51.395491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:51.396090 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:51.398985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:51.399729 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:51.406280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.006s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:51.443835 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Getting quotas for project f37b53a21b19482b8dcac6c38505a50b. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:03:51.458586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Getting quotas for user 1e515c9c8dcf4a1aa7dd74fb7655adf4 and project f37b53a21b19482b8dcac6c38505a50b. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:03:51.483666 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:03:51.489909 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:03:51.620658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-504fdeb4-d169-412a-a1ac-360d03b9fac7 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.460504 Sep 25 01:03:51.622794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 430/848] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:03:50 2022] POST /compute/v2.1/servers => generated 402 bytes in 1465 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 01:03:51.640391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4789b9dc-b6f2-49e2-8864-bc57db0a357e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:51.641736 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4789b9dc-b6f2-49e2-8864-bc57db0a357e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:51.794036 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4789b9dc-b6f2-49e2-8864-bc57db0a357e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:51.818133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4789b9dc-b6f2-49e2-8864-bc57db0a357e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1236 microversion: 2.1 time: 0.183183 Sep 25 01:03:51.840227 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 419/849] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:51 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1236 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:03:52.863812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:52.864603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:52.878681 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:52.879839 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:52.880580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:52.881590 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:52.882305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:52.882908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:52.992004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:53.006388 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3f263f5f-e7b8-4a66-8114-82788c98f08b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1232 microversion: 2.1 time: 0.148196 Sep 25 01:03:53.007801 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 431/850] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:52 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1232 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:03:54.037635 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:54.038949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:54.052372 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:54.053224 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:54.054049 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:54.055331 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:54.056178 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:54.056971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:54.154422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:54.163649 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-aec46421-0a3e-4494-93bd-851aad635028 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1288 microversion: 2.1 time: 0.134162 Sep 25 01:03:54.164244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 420/851] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:54 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1288 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:03:55.204446 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:55.205513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:55.213949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:55.214553 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:55.215137 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:55.215949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:55.216513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:55.217360 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:55.325345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:55.340067 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e4de425a-1c44-4398-ae76-5f058fe7559a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1294 microversion: 2.1 time: 0.139072 Sep 25 01:03:55.341313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 432/852] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:55 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1294 bytes in 141 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:03:56.380552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:56.381650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:56.390284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:56.390944 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:56.391613 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:56.392562 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:56.393635 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:56.394136 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:56.592610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:56.608278 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8a65eb44-d619-448b-add8-5f97f295d7e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1362 microversion: 2.1 time: 0.234653 Sep 25 01:03:56.609661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 421/853] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:56 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1362 bytes in 237 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:03:57.639266 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:57.640350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:57.650663 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:57.651500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:57.652229 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:57.653675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:57.654451 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:57.655171 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:57.849844 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:57.860413 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8f2ec651-fc9f-4d21-92fa-c43248d59e42 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1362 microversion: 2.1 time: 0.227961 Sep 25 01:03:57.861252 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 433/854] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:57 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1362 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:03:58.888335 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:03:58.890000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:03:58.908939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:58.909656 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:58.910106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:58.910832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:58.911300 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:58.911729 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:59.180915 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:03:59.197346 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c098e923-722d-4ff2-8f16-c1e66579c697 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1362 microversion: 2.1 time: 0.312907 Sep 25 01:03:59.198670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 422/855] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:03:58 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1362 bytes in 315 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:03:59.274977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-d79cebcc-3bda-435e-b6a9-068d532c7b15 req-b9b1dfb4-54c7-45aa-a5b9-cbc886922985 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1de127ae-8886-4d15-a6be-7b68902ccde8", "tag": "c71c9659-825c-4f3d-b945-2c964a1f67a7"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:03:59.296251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d79cebcc-3bda-435e-b6a9-068d532c7b15 req-b9b1dfb4-54c7-45aa-a5b9-cbc886922985 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:03:59.298260 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d79cebcc-3bda-435e-b6a9-068d532c7b15 req-b9b1dfb4-54c7-45aa-a5b9-cbc886922985 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:03:59.299023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d79cebcc-3bda-435e-b6a9-068d532c7b15 req-b9b1dfb4-54c7-45aa-a5b9-cbc886922985 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:03:59.312311 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-d79cebcc-3bda-435e-b6a9-068d532c7b15 req-b9b1dfb4-54c7-45aa-a5b9-cbc886922985 service nova] Creating event network-changed:c71c9659-825c-4f3d-b945-2c964a1f67a7 for instance 1de127ae-8886-4d15-a6be-7b68902ccde8 on hv-cinder-maste Sep 25 01:03:59.326102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-d79cebcc-3bda-435e-b6a9-068d532c7b15 req-b9b1dfb4-54c7-45aa-a5b9-cbc886922985 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062500 Sep 25 01:03:59.328970 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 434/856] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:03:59 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 67 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:04:00.245460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:00.246096 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:00.253312 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:00.253813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:00.254252 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:00.254970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:00.255422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:00.255851 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:00.440094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:00.448243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4a0a9c03-a2a5-4aff-acf3-48db7b37ac70 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1362 microversion: 2.1 time: 0.206723 Sep 25 01:04:00.448838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 423/857] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:00 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1362 bytes in 208 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:04:01.471757 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:01.477778 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:01.478926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:01.481290 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=6 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:01.489516 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:01.490027 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:01.490470 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:01.491192 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:01.491658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:01.492091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:01.666090 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:01.676462 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-53f92164-e9dd-4510-b3e0-a3e0d2560b31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1362 microversion: 2.1 time: 0.207560 Sep 25 01:04:01.677397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 435/858] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:01 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1362 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:02.474731 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-TestVolumeBootPattern-server-454691508", "imageRef": "", "flavorRef": "452", "key_name": "tempest-TestVolumeBootPattern-1527375203", "security_groups": [{"name": "tempest-secgroup-smoke-1834758444"}], "block_device_mapping_v2": [{"uuid": "aa88d65d-cf36-465c-a62d-5cebf2c1d5e4", "source_type": "volume", "destination_type": "volume", "boot_index": 0, "delete_on_termination": false}], "networks": [{"uuid": "31fa7821-6345-4f48-b070-9ceab6d12adf"}]}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:04:02.500454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=6 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:02.667798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] validate_networks() for [('31fa7821-6345-4f48-b070-9ceab6d12adf', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:04:02.708297 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:02.710713 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:02.724868 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:02.725870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:02.727025 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:02.731274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:02.732599 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:02.733540 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:03.030092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:03.040830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-884da860-7571-4541-ab0e-c0bbdacc5aed tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1362 microversion: 2.1 time: 0.341712 Sep 25 01:04:03.042001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 436/859] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:02 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1362 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:04:03.369883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:04:03.370422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:04:03.371052 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:04:03.371605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:04:03.372347 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:04:03.385314 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:03.386219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:03.386958 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:03.387933 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:03.388579 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:03.389600 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:03.423130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Getting quotas for project 46f19fd9ad1e4880961d1eee75883687. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:04:03.436739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Getting quotas for user c19b10ebc31244dbb2b1539d94ac281d and project 46f19fd9ad1e4880961d1eee75883687. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:04:03.522381 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:04:03.524816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=False,deleted=,deleted_at=,destination_type='volume',device_name=None,device_type=None,disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id=None,instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='volume',tag=None,updated_at=,uuid=,volume_id='aa88d65d-cf36-465c-a62d-5cebf2c1d5e4',volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:04:03.709946 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a9fe558c-2bef-47b6-9ea2-f8c3a3608763 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 406 microversion: 2.1 time: 1.243315 Sep 25 01:04:03.710998 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 424/860] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:04:02 2022] POST /compute/v2.1/servers => generated 406 bytes in 1246 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 01:04:03.727990 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-060230f6-bec3-4e7d-8e6f-0db35d925d04 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:03.729244 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-060230f6-bec3-4e7d-8e6f-0db35d925d04 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:03.797219 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-060230f6-bec3-4e7d-8e6f-0db35d925d04 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:03.805443 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-060230f6-bec3-4e7d-8e6f-0db35d925d04 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1094 microversion: 2.1 time: 0.085588 Sep 25 01:04:03.807882 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 437/861] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:03 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1094 bytes in 89 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:04.069283 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:04.070860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:04.083419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:04.084006 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:04.084461 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:04.085303 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:04.088506 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:04.088995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.003s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:04.281971 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-cdfb4215-1703-45be-ae7d-3e3fefd2829b req-17f4fa27-4ffb-444d-a329-2ce27a39cfac service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1de127ae-8886-4d15-a6be-7b68902ccde8", "name": "network-vif-plugged", "status": "completed", "tag": "c71c9659-825c-4f3d-b945-2c964a1f67a7"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:04:04.303348 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-cdfb4215-1703-45be-ae7d-3e3fefd2829b req-17f4fa27-4ffb-444d-a329-2ce27a39cfac service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:04.303952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-cdfb4215-1703-45be-ae7d-3e3fefd2829b req-17f4fa27-4ffb-444d-a329-2ce27a39cfac service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:04.304694 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-cdfb4215-1703-45be-ae7d-3e3fefd2829b req-17f4fa27-4ffb-444d-a329-2ce27a39cfac service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:04.321807 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-cdfb4215-1703-45be-ae7d-3e3fefd2829b req-17f4fa27-4ffb-444d-a329-2ce27a39cfac service nova] Creating event network-vif-plugged:c71c9659-825c-4f3d-b945-2c964a1f67a7 for instance 1de127ae-8886-4d15-a6be-7b68902ccde8 on hv-cinder-maste Sep 25 01:04:04.331060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-cdfb4215-1703-45be-ae7d-3e3fefd2829b req-17f4fa27-4ffb-444d-a329-2ce27a39cfac service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.055128 Sep 25 01:04:04.333508 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 438/862] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:04:04 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:04:04.351266 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:04.364935 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c6c56705-0a2f-4dc1-b4f5-4c8f35f01c9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1362 microversion: 2.1 time: 0.303133 Sep 25 01:04:04.365860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 425/863] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:04 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1362 bytes in 306 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:04.843577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:04.844313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:04.851698 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:04.852196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:04.852639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:04.853406 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:04.853874 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:04.854307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:04.922931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:04.935939 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f87f362a-e893-4e59-91c4-a6192f11738b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1136 microversion: 2.1 time: 0.097025 Sep 25 01:04:04.936671 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 439/864] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:04 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1136 bytes in 99 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:04:05.403646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:05.404338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:05.411738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:05.412440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:05.412893 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:05.413765 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:05.414339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:05.414792 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:05.574414 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:05.583456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3f087363-260e-44af-b6fa-b2d22a793433 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1362 microversion: 2.1 time: 0.183810 Sep 25 01:04:05.585581 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 426/865] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:05 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1362 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:05.960754 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:05.961557 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:05.968820 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:05.969377 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:05.969864 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:05.971151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:05.971619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:05.972057 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:06.043023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:06.052378 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e06b3ac5-3caa-47c4-99a1-bca28ced01ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1192 microversion: 2.1 time: 0.095655 Sep 25 01:04:06.057691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 440/866] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:05 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1192 bytes in 97 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:06.608214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:06.609403 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:06.619266 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:06.619881 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:06.620418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:06.621339 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:06.621849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:06.622461 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:06.776652 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:06.785445 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e61ff87b-cf53-4f99-bb14-a60b88a2f81d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1547 microversion: 2.1 time: 0.182541 Sep 25 01:04:06.786040 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 427/867] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:06 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1547 bytes in 184 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:04:06.812534 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:06.813460 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:06.825622 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:06.826589 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:06.827376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:06.828667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:06.829511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:06.831738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:06.956322 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c3900e66-b623-45a9-b585-6b89f79576d2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/os-interface" status: 200 len: 284 microversion: 2.1 time: 0.150808 Sep 25 01:04:06.964194 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 441/868] 10.222.0.52 () {58 vars in 1275 bytes} [Sun Sep 25 01:04:06 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/os-interface => generated 284 bytes in 156 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:04:07.095282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:07.096954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:07.123325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:07.129364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.006s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:07.130707 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:07.132391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:07.137181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.005s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:07.138673 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:07.286657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:07.298537 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0cdf1143-df87-4238-8ac2-43634e59367b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1200 microversion: 2.1 time: 0.209497 Sep 25 01:04:07.313300 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 428/869] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:07 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1200 bytes in 214 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:08.335556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:08.337295 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:08.348965 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:08.350184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:08.350630 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:08.351617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:08.352232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:08.352851 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:08.948028 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:08.957500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-688e8b96-f509-4c6a-8b2c-a424e49012fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1278 microversion: 2.1 time: 0.627282 Sep 25 01:04:08.958100 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 442/870] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:08 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1278 bytes in 629 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:04:09.255649 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-c69a4fa2-b44a-4869-b355-fbaa437a8a69 req-363603d7-5b72-4802-93b7-59452a492066 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1de127ae-8886-4d15-a6be-7b68902ccde8", "tag": "c71c9659-825c-4f3d-b945-2c964a1f67a7"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:04:09.286291 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c69a4fa2-b44a-4869-b355-fbaa437a8a69 req-363603d7-5b72-4802-93b7-59452a492066 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:09.287979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c69a4fa2-b44a-4869-b355-fbaa437a8a69 req-363603d7-5b72-4802-93b7-59452a492066 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:09.289310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c69a4fa2-b44a-4869-b355-fbaa437a8a69 req-363603d7-5b72-4802-93b7-59452a492066 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:09.309533 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-c69a4fa2-b44a-4869-b355-fbaa437a8a69 req-363603d7-5b72-4802-93b7-59452a492066 service nova] Creating event network-changed:c71c9659-825c-4f3d-b945-2c964a1f67a7 for instance 1de127ae-8886-4d15-a6be-7b68902ccde8 on hv-cinder-maste Sep 25 01:04:09.317021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-c69a4fa2-b44a-4869-b355-fbaa437a8a69 req-363603d7-5b72-4802-93b7-59452a492066 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.068470 Sep 25 01:04:09.317708 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 429/871] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:04:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 71 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:04:09.986344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:09.987195 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:09.995012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:09.995836 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:09.996570 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:09.997586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:09.998333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:09.998835 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:10.199877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:10.210513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f27e002b-71b3-4ce6-b966-9ba9f28c9d7a tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.229212 Sep 25 01:04:10.211202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 443/872] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:09 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:04:10.332971 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-6824ecbb-567e-4c15-9fd7-f5bf87896644 req-b86ace9b-098e-4f07-b228-0f6e528e9bc1 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4814bc8c-42fd-4665-9123-2feba2f7887f", "tag": "9d97f0d8-3c7b-4ac3-8d16-6a80eec13764"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:04:10.353507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-6824ecbb-567e-4c15-9fd7-f5bf87896644 req-b86ace9b-098e-4f07-b228-0f6e528e9bc1 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:10.354716 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-6824ecbb-567e-4c15-9fd7-f5bf87896644 req-b86ace9b-098e-4f07-b228-0f6e528e9bc1 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:10.355485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-6824ecbb-567e-4c15-9fd7-f5bf87896644 req-b86ace9b-098e-4f07-b228-0f6e528e9bc1 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:10.377636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-6824ecbb-567e-4c15-9fd7-f5bf87896644 req-b86ace9b-098e-4f07-b228-0f6e528e9bc1 service nova] Creating event network-changed:9d97f0d8-3c7b-4ac3-8d16-6a80eec13764 for instance 4814bc8c-42fd-4665-9123-2feba2f7887f on hv-cinder-maste Sep 25 01:04:10.386406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-6824ecbb-567e-4c15-9fd7-f5bf87896644 req-b86ace9b-098e-4f07-b228-0f6e528e9bc1 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.062357 Sep 25 01:04:10.387576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 430/873] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:04:10 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 65 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:04:11.240763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:11.241590 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:11.249485 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:11.250160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:11.250706 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:11.252282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:11.253150 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:11.253688 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:11.426095 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:11.435905 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8074c629-7066-4873-bd90-a2560f984b3d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.200567 Sep 25 01:04:11.436653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 444/874] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:11 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:12.461885 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:12.463115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:12.470903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:12.471375 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:12.472019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:12.472781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:12.473377 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:12.473851 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:12.640529 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:12.649725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-503d008c-88b7-441b-bc0f-2cd4a4adbe71 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.192471 Sep 25 01:04:12.650324 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 431/875] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:12 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:13.691592 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:13.692621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:13.705227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:13.706224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:13.707007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:13.708347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:13.709183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:13.709988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:13.807353 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:13.948073 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:13.960204 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ef9e3c47-731a-4f05-ab1e-7ffdbe26726f tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.276717 Sep 25 01:04:13.964197 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 445/876] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:13 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 282 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:14.991241 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:14.991904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:14.994057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:14.994776 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=6 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:15.002963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:15.003456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:15.003898 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:15.004623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:15.005082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:15.005552 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:15.172297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:15.181098 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9712e7ee-6e93-4334-b947-cbd12b7df5ae tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.194102 Sep 25 01:04:15.181752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 432/877] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:14 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:16.204906 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:16.206603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:16.214346 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:16.214840 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:16.215276 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:16.215993 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:16.216447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:16.216869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:16.382842 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:16.391601 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3adb1c0c-d102-46fc-9d0a-cf6ded70fb6b tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.191100 Sep 25 01:04:16.392787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 446/878] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:16 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 193 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:04:17.429370 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:17.430046 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:17.432603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=55,nova_api:INSERT=4 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:17.438827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:17.439328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:17.439787 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:17.440540 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:17.440982 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:17.441455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:17.445437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=67 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:17.605592 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:17.615181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c0f2b3fd-36c7-4adb-879c-7f926658f076 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.189737 Sep 25 01:04:17.615992 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 433/879] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:17 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:18.639089 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:18.639725 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:18.647306 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:18.647798 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:18.648241 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:18.648987 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:18.649499 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:18.649934 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:18.848389 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:18.857603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9e7c99c5-03f5-4c44-bb1f-d40fe3d5a845 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.222861 Sep 25 01:04:18.858217 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 447/880] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:18 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 224 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:19.902406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:19.903546 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:19.915239 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:19.916196 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:19.917295 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:19.918658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:19.919786 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:19.920594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:20.111818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:20.120923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-1b9c4bc6-0089-437d-bbb8-b9ccee74c408 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.227137 Sep 25 01:04:20.121580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 434/881] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:19 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 230 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:04:21.146602 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:21.147346 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:21.150144 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=52,nova_api:INSERT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:21.156172 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:21.156700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:21.157792 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:21.158699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:21.159296 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:21.159871 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:21.164567 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=89 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:21.335508 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:21.346604 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b9866115-ac35-460f-b456-9e1211e0ef87 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.203923 Sep 25 01:04:21.347312 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 448/882] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:21 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:22.370593 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:22.371388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:22.379163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:22.379661 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:22.380108 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:22.380828 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:22.381313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:22.381750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:22.563409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:22.577869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-71306f59-f667-49e9-bc6b-c4ad18180275 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.211183 Sep 25 01:04:22.578464 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 435/883] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:22 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 213 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:23.603426 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:23.604138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:23.613107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:23.613637 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:23.614074 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:23.614789 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:23.615241 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:23.615662 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:23.800834 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:23.809406 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8fc1701e-14db-47a3-8168-fb07bff3a480 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.209991 Sep 25 01:04:23.810001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 449/884] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:23 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:24.833936 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:24.835060 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:24.843929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:24.844541 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:24.845418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:24.846471 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:24.847083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:24.847611 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:25.044747 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-a391d6f2-f12a-4ddd-8a71-8375a58f2913 req-db0204b0-8e7a-489c-bcbb-ec89bf688511 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4814bc8c-42fd-4665-9123-2feba2f7887f", "name": "network-vif-plugged", "status": "completed", "tag": "9d97f0d8-3c7b-4ac3-8d16-6a80eec13764"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:04:25.056784 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:25.068138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-41a297ae-9988-4822-b038-a1dc7f93b084 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.238927 Sep 25 01:04:25.069174 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 436/885] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:24 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 241 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:25.070213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-a391d6f2-f12a-4ddd-8a71-8375a58f2913 req-db0204b0-8e7a-489c-bcbb-ec89bf688511 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:25.070800 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-a391d6f2-f12a-4ddd-8a71-8375a58f2913 req-db0204b0-8e7a-489c-bcbb-ec89bf688511 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:25.071334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-a391d6f2-f12a-4ddd-8a71-8375a58f2913 req-db0204b0-8e7a-489c-bcbb-ec89bf688511 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:25.086198 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-a391d6f2-f12a-4ddd-8a71-8375a58f2913 req-db0204b0-8e7a-489c-bcbb-ec89bf688511 service nova] Creating event network-vif-plugged:9d97f0d8-3c7b-4ac3-8d16-6a80eec13764 for instance 4814bc8c-42fd-4665-9123-2feba2f7887f on hv-cinder-maste Sep 25 01:04:25.092117 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-a391d6f2-f12a-4ddd-8a71-8375a58f2913 req-db0204b0-8e7a-489c-bcbb-ec89bf688511 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052995 Sep 25 01:04:25.092721 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 450/886] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:04:25 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:04:26.092179 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:26.092799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:26.099870 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:26.100534 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:26.101045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:26.101965 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:26.102482 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:26.102969 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:26.269382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:26.277837 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-83a5250b-7916-40ef-b6b8-e28678be350e tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1266 microversion: 2.1 time: 0.189429 Sep 25 01:04:26.278946 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 437/887] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:26 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1266 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:27.317596 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:27.318204 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:27.324903 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:27.325420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:27.325857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:27.326573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:27.327027 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:27.327455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:27.487351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:27.500266 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0c48a889-6012-4964-80ab-1314cac67fd9 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1453 microversion: 2.1 time: 0.187828 Sep 25 01:04:27.511232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 451/888] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:27 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1453 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:27.527783 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:27.528406 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:27.536029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:27.536523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:27.536961 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:27.537725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:27.538181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:27.538612 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:27.687060 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:27.695524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-27a16088-bae8-4309-8971-8a07bee360d3 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1453 microversion: 2.1 time: 0.171625 Sep 25 01:04:27.696112 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 438/889] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:27 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1453 bytes in 173 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:04:30.026923 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-2a7f25ca-9ce0-4466-9b9c-077025335fd0 req-f6f1e2e3-7389-4b8d-b6bb-81f76c8c07d3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4814bc8c-42fd-4665-9123-2feba2f7887f", "tag": "9d97f0d8-3c7b-4ac3-8d16-6a80eec13764"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:04:30.040112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2a7f25ca-9ce0-4466-9b9c-077025335fd0 req-f6f1e2e3-7389-4b8d-b6bb-81f76c8c07d3 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:30.040762 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2a7f25ca-9ce0-4466-9b9c-077025335fd0 req-f6f1e2e3-7389-4b8d-b6bb-81f76c8c07d3 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:30.041394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2a7f25ca-9ce0-4466-9b9c-077025335fd0 req-f6f1e2e3-7389-4b8d-b6bb-81f76c8c07d3 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:30.054869 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-2a7f25ca-9ce0-4466-9b9c-077025335fd0 req-f6f1e2e3-7389-4b8d-b6bb-81f76c8c07d3 service nova] Creating event network-changed:9d97f0d8-3c7b-4ac3-8d16-6a80eec13764 for instance 4814bc8c-42fd-4665-9123-2feba2f7887f on hv-cinder-maste Sep 25 01:04:30.060302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-2a7f25ca-9ce0-4466-9b9c-077025335fd0 req-f6f1e2e3-7389-4b8d-b6bb-81f76c8c07d3 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.037103 Sep 25 01:04:30.061104 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 452/890] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:04:30 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:04:45.850369 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:45.851002 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:45.852920 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=11 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:45.853827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=35 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:45.861691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:45.862190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:45.862637 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:45.863357 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:45.863817 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:45.864386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:45.896189 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-db86ec78-407b-438a-885b-5cc585a164c6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/ips" status: 200 len: 144 microversion: 2.1 time: 0.050020 Sep 25 01:04:45.897766 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 439/891] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 01:04:45 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/ips => generated 144 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:04:50.815886 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=9 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:50.816828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=21 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:50.821248 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "e21ef985-5157-4b4a-ae45-1bd0ed2df8d7"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:04:50.823760 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:50.831400 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:50.831938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:50.832472 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:50.833230 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:50.833794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:50.834359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:51.542550 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e022c45b-87ef-4213-93fa-7e63c8a6ea9c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.727299 Sep 25 01:04:51.543736 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 453/892] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 01:04:50 2022] POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/os-volume_attachments => generated 194 bytes in 730 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:04:59.241307 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f4e745c1-3a63-40a6-aecf-a868c78a42cd tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:04:59.243918 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:59.244638 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=4 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:04:59.251600 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f4e745c1-3a63-40a6-aecf-a868c78a42cd tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:59.252148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f4e745c1-3a63-40a6-aecf-a868c78a42cd tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:59.252645 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f4e745c1-3a63-40a6-aecf-a868c78a42cd tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:59.323737 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f4e745c1-3a63-40a6-aecf-a868c78a42cd tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Going to try to stop instance {{(pid=120450) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 25 01:04:59.380714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f4e745c1-3a63-40a6-aecf-a868c78a42cd tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action" status: 202 len: 0 microversion: 2.1 time: 0.143854 Sep 25 01:04:59.381344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 440/893] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:04:59 2022] POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action => generated 0 bytes in 145 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:04:59.391958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:04:59.393124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:04:59.401224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:59.402087 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:59.402644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:59.403701 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:04:59.404335 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:04:59.405535 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:04:59.599507 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:04:59.610028 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-02cfc57e-66cd-4876-b5eb-5a292d6707d6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.223603 Sep 25 01:04:59.610626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 454/894] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:04:59 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:00.636069 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:00.636681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:00.644689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:00.645190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:00.645674 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:00.646560 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:00.647056 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:00.647481 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:00.805670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:00.815494 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-76342d16-d56a-479f-b463-599611441929 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.184733 Sep 25 01:05:00.816186 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 441/895] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:00 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:01.841160 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:01.841960 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:01.851005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:01.851573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:01.852304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:01.853099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:01.853648 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:01.854162 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:02.022664 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:02.034256 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-68ca1a99-eadb-471c-b144-287d590fbfc2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.197029 Sep 25 01:05:02.034999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 455/896] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:01 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 199 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:03.060165 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:03.060814 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:03.068161 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:03.068697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:03.069282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:03.070191 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:03.070748 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:03.071225 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:03.225959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:03.236928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-08dd25b7-3add-4664-a1bf-4df7924a30cc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.180575 Sep 25 01:05:03.237576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 442/897] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:03 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:04.262086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:04.262699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:04.270386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:04.270899 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:04.271348 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:04.272076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:04.272537 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:04.272966 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:04.469616 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:04.482777 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d93d9621-4065-40f9-986c-f120d5edf030 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.224581 Sep 25 01:05:04.483661 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 456/898] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:04 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 226 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:05.210840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Action: 'action', calling method: >, body: {"shelve": {}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:05:05.211675 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:05.219054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:05.219548 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:05.219985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:05.220696 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:05.221150 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:05.221604 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:05.248696 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lazy-loading 'flavor' on Instance uuid eb66c484-73f0-441b-a575-f506a0caca8a {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:05:05.338039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9e727491-9035-4810-af66-f9fa4fa06906 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "POST /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a/action" status: 202 len: 0 microversion: 2.1 time: 0.130875 Sep 25 01:05:05.338856 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 443/899] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:05:05 2022] POST /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a/action => generated 0 bytes in 133 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 01:05:05.348785 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:05.349453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:05.356647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:05.357151 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:05.357645 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:05.359561 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:05.359903 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:05.361158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:05.511574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:05.512545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:05.519446 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:05.519944 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:05.520395 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:05.521136 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:05.521632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:05.522067 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:05.559555 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:05.569476 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5ae56263-9dcc-4965-b125-cb04bc9797d8 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.224793 Sep 25 01:05:05.570460 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 457/900] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:05 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 227 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:05.719127 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:05.728639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-dd9c3757-3e1b-4724-934a-1d5a7107c7fe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.223752 Sep 25 01:05:05.729221 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 444/901] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:05 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 225 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:06.609857 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:06.610549 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:06.617402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:06.617889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:06.618319 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:06.619020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:06.619573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:06.620207 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:06.758925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:06.759702 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:06.767978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:06.768471 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:06.768909 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:06.769664 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:06.770106 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:06.770535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:06.778559 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:06.790736 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5b2b105c-39b1-43a9-915d-67aad30574e7 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.185102 Sep 25 01:05:06.791239 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 458/902] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:06 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:06.948430 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:06.958600 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c2c165de-239c-4ffb-abb6-ccffb3087bff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.204481 Sep 25 01:05:06.959197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 445/903] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:06 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:07.816720 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:07.817525 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:07.825773 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:07.826264 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:07.826697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:07.828499 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:07.829107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:07.829567 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:07.982192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:07.982844 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:07.990531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:07.991305 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:07.991821 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:07.992632 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:07.993163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:07.993671 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:08.000128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:08.016253 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a88821ff-4800-4b52-b280-152f7b03bd35 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.203154 Sep 25 01:05:08.017378 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 459/904] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:07 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:08.302195 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:08.310957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6f2361e4-72cc-4379-b111-cb9b86c78c0c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.332671 Sep 25 01:05:08.311590 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 446/905] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:07 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 334 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:09.053378 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:09.054058 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:09.063475 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:09.063966 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:09.064413 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:09.065129 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:09.065612 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:09.066035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:09.210144 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:09.219392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6d71da0f-c043-495c-b0f3-bded0837c35b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.169848 Sep 25 01:05:09.219987 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 460/906] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:09 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:09.335239 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:09.336038 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:09.343262 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:09.343767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:09.344215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:09.344936 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:09.345431 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:09.345866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:09.491444 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:09.506636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-98f8f56d-0854-473e-bdc7-94abfbda4f1a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1721 microversion: 2.1 time: 0.174979 Sep 25 01:05:09.507699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 447/907] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:09 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1721 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:10.245124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:10.245803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:10.253013 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:10.253527 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:10.253960 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:10.255111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:10.255111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:10.255502 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:10.404879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:10.414184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2514ea0e-ae89-4c6d-b76a-8ad6e5fa1f85 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.172884 Sep 25 01:05:10.414765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 461/908] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:10 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:10.543486 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:10.544287 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:10.552643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:10.553053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:10.553511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:10.554226 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:10.554685 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:10.555112 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:10.732437 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:10.747104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b4ea2ace-6c02-44ba-83bf-3b2c8124c1cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1698 microversion: 2.1 time: 0.207655 Sep 25 01:05:10.747701 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 448/909] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:10 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1698 bytes in 209 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:10.779703 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ceeffa24-4530-4b5c-a70e-03f87cd5e196 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:05:10.787333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ceeffa24-4530-4b5c-a70e-03f87cd5e196 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:10.787833 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ceeffa24-4530-4b5c-a70e-03f87cd5e196 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:10.788301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ceeffa24-4530-4b5c-a70e-03f87cd5e196 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:10.855028 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ceeffa24-4530-4b5c-a70e-03f87cd5e196 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Going to try to start instance {{(pid=120451) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 25 01:05:10.907563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ceeffa24-4530-4b5c-a70e-03f87cd5e196 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action" status: 202 len: 0 microversion: 2.1 time: 0.131927 Sep 25 01:05:10.908222 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 462/910] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:05:10 2022] POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action => generated 0 bytes in 134 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:05:10.917644 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:10.918780 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:10.927544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:10.928393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:10.928986 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:10.930119 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:10.930886 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:10.931462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:11.180025 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:11.190813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3ca27ab5-34a8-4493-a815-d9dd402a271e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1707 microversion: 2.1 time: 0.277641 Sep 25 01:05:11.191566 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 449/911] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:10 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1707 bytes in 280 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:11.440750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:11.441668 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:11.448912 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:11.449447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:11.449892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:11.450607 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:11.451066 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:11.451492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:11.628338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:11.644856 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b37f4e62-83a9-47a3-b952-12aaa735cf3c tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.208533 Sep 25 01:05:11.646373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 463/912] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:11 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 211 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:12.216064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:12.216647 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:12.223989 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:12.224493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:12.224939 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:12.225700 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:12.226189 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:12.226618 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:12.401405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:12.410604 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-21a96828-10ef-4ff6-ad63-9dd7197f9d68 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1707 microversion: 2.1 time: 0.198864 Sep 25 01:05:12.411202 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 450/913] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:12 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1707 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:12.686452 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:12.687762 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:12.695127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:12.695704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:12.696248 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:12.697017 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:12.697714 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:12.698280 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:12.859770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:12.868193 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-bec30b3f-9281-4aef-80a0-04486ee1ee09 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.185795 Sep 25 01:05:12.868786 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 464/914] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:12 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:13.435635 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:13.436295 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:13.444005 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:13.444505 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:13.444951 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:13.445704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:13.446170 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:13.446599 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:13.623883 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:13.636227 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-f22f28fc-cdc3-426a-9243-3f6e01991fe5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1711 microversion: 2.1 time: 0.204632 Sep 25 01:05:13.636852 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 451/915] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:13 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1711 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:13.896509 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:13.897712 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:13.909204 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:13.910201 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:13.910948 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:13.912174 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:13.912991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:13.913805 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:14.093849 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:14.103093 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-497a12ef-9db5-4dee-9f88-cf388a5b5a53 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.214128 Sep 25 01:05:14.103683 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 465/916] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:13 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 216 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:15.133982 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:15.135072 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:15.147045 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:15.147936 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:15.148752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:15.150030 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:15.150896 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:15.151707 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:15.306999 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:15.317832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-971029e1-9e48-4f09-85bb-042cce481842 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.190575 Sep 25 01:05:15.318427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 452/917] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:15 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:16.356995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:16.357684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:16.366357 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:16.367264 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:16.368077 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:16.369479 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:16.370283 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:16.371024 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:16.527870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:16.536913 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-605fd198-074c-4aa1-afc0-7112e518f96b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.183885 Sep 25 01:05:16.537695 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 466/918] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:16 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:17.560873 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:17.561691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:17.569888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:17.570489 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:17.571000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:17.571767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:17.572291 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:17.572754 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:17.596097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:17.731284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:17.740487 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-30c5bf1d-7b2f-4c98-a3cb-9e15bc1a1379 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.183649 Sep 25 01:05:17.741069 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 453/919] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:17 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:18.782499 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:18.783107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:18.790127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:18.790609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:18.791041 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:18.791840 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:18.792288 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:18.792710 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:18.799408 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=95,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:18.942283 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:18.950697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-20d196a5-cb99-4cba-8ff4-f19b411cc680 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1569 microversion: 2.1 time: 0.173128 Sep 25 01:05:18.951289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 467/920] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:18 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1569 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:05:18.975458 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Action: 'action', calling method: >, body: {"unshelve": null} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:05:18.976097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:18.983026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:18.983546 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:18.984366 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:18.985074 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:18.985572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:18.986004 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:19.019198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Unshelving instance with old availability_zone "None" to new availability_zone "not provided" and host "None". {{(pid=120450) unshelve /opt/stack/nova/nova/compute/api.py:4593}} Sep 25 01:05:19.101732 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-12c3ab74-0024-4a01-bd85-292d99b526eb tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "POST /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a/action" status: 202 len: 0 microversion: 2.1 time: 0.130265 Sep 25 01:05:19.102546 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 454/921] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:05:18 2022] POST /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a/action => generated 0 bytes in 132 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:05:19.112905 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:19.113556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:19.120430 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:19.120920 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:19.121387 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:19.122091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:19.122879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:19.123042 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:19.351708 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:19.363758 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f165c17f-a9a7-4d3f-b9f4-bcc197faa634 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1517 microversion: 2.1 time: 0.256480 Sep 25 01:05:19.364884 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 468/922] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:19 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1517 bytes in 259 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:20.398219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:20.398856 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:20.405275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:20.405791 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:20.406234 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:20.406945 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:20.407409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:20.407840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:20.552934 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:20.560984 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5e92c0dc-606e-44c6-ae97-0b164ef4cc6d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1519 microversion: 2.1 time: 0.167417 Sep 25 01:05:20.562474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 455/923] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:20 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1519 bytes in 170 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:21.211875 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-8a2819fa-a3b9-422c-aa2e-1a0803b6c5a8 req-798901b7-1080-4840-889d-178bc3acf0fd service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb66c484-73f0-441b-a575-f506a0caca8a", "name": "network-vif-unplugged", "status": "completed", "tag": "7589db85-95c8-412c-a8f7-5db156ef7906"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:05:21.219486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=33 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:21.224915 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8a2819fa-a3b9-422c-aa2e-1a0803b6c5a8 req-798901b7-1080-4840-889d-178bc3acf0fd service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:21.225439 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8a2819fa-a3b9-422c-aa2e-1a0803b6c5a8 req-798901b7-1080-4840-889d-178bc3acf0fd service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:21.226302 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8a2819fa-a3b9-422c-aa2e-1a0803b6c5a8 req-798901b7-1080-4840-889d-178bc3acf0fd service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:21.239998 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-8a2819fa-a3b9-422c-aa2e-1a0803b6c5a8 req-798901b7-1080-4840-889d-178bc3acf0fd service nova] Creating event network-vif-unplugged:7589db85-95c8-412c-a8f7-5db156ef7906 for instance eb66c484-73f0-441b-a575-f506a0caca8a on hv-cinder-maste Sep 25 01:05:21.245510 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-8a2819fa-a3b9-422c-aa2e-1a0803b6c5a8 req-798901b7-1080-4840-889d-178bc3acf0fd service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 185 microversion: 2.1 time: 0.037266 Sep 25 01:05:21.246126 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 469/924] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:05:21 2022] POST /compute/v2.1/os-server-external-events => generated 185 bytes in 39 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:05:21.585601 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:21.586197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:21.593826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:21.594319 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:21.594755 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:21.596135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:21.596604 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:21.597035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:21.840200 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:21.849344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-59725a50-8d5d-4919-8dda-3958b888ffae tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.267643 Sep 25 01:05:21.849968 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 456/925] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:21 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 269 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:22.876488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:22.877155 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:22.884677 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:22.885206 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:22.885697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:22.886507 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:22.886999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:22.887430 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:23.035291 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:23.043526 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-25be2e34-3e7f-423b-99e6-79ca5bdff82f tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.172352 Sep 25 01:05:23.044104 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 470/926] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:22 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 174 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:24.068901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:24.069834 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:24.076531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:24.077059 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:24.077926 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:24.078271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:24.078728 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:24.079169 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:24.236093 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:24.244502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e954593f-d727-4658-90f7-cbca260b3666 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.180091 Sep 25 01:05:24.245310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 457/927] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:24 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:25.270188 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:25.271067 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:25.278047 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:25.278550 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:25.278993 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:25.279730 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:25.280709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:25.281227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:25.449513 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:25.459956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-60fd0b06-2ed7-4135-9ce5-fc86585207ce tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.193725 Sep 25 01:05:25.460555 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 471/928] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:25 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:26.491507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:26.492197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:26.500824 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:26.501355 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:26.501795 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:26.502524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:26.502999 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:26.503420 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:26.658623 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:26.667492 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ebb463db-a2d8-4ffc-bd4b-b74e22eb0b9b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.179851 Sep 25 01:05:26.668097 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 458/929] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:26 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:27.693048 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:27.693798 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:27.700299 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:27.700798 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:27.701243 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:27.702061 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:27.703135 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:27.703573 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:27.857339 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:27.868895 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f0415ad3-ee5a-4cd3-8bec-eba3a867e835 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.179965 Sep 25 01:05:27.869690 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 472/930] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:27 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:28.893391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:28.893962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:28.901019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:28.901743 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:28.902248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:28.903114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:28.903647 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:28.904189 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:29.059055 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:29.067972 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c28d1077-93d5-48e6-bd56-b5e30811a96a tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.178761 Sep 25 01:05:29.068580 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 459/931] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:28 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:30.092071 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:30.093402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:30.100454 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:30.100930 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:30.101405 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:30.102137 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:30.102591 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:30.103012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:30.261248 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:30.271512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-aae67070-5668-4291-a56c-26042ca267f5 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.183606 Sep 25 01:05:30.272105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 473/932] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:30 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:31.296586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:31.298333 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:31.300640 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:31.306275 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:31.306763 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:31.307202 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:31.308204 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:31.308662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:31.309086 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:31.460832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:31.469607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-0fa21332-021a-452a-8e3d-db172372e918 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.177195 Sep 25 01:05:31.470185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 460/933] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:31 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:32.494613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:32.495520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:32.510219 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:32.510709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:32.511141 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:32.511853 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:32.512448 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:32.512892 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:32.660716 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:32.668745 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a39ff6fd-a4b0-4cef-a164-f108c797c3dc tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.178009 Sep 25 01:05:32.669351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 474/934] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:32 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:33.314260 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-aee4ce58-82dc-491e-a9f3-cbcadcbe535d req-c694f196-0275-4b40-b177-5649dea678b3 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb66c484-73f0-441b-a575-f506a0caca8a", "name": "network-vif-plugged", "status": "completed", "tag": "7589db85-95c8-412c-a8f7-5db156ef7906"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:05:33.325015 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-aee4ce58-82dc-491e-a9f3-cbcadcbe535d req-c694f196-0275-4b40-b177-5649dea678b3 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:33.325530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-aee4ce58-82dc-491e-a9f3-cbcadcbe535d req-c694f196-0275-4b40-b177-5649dea678b3 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:33.325983 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-aee4ce58-82dc-491e-a9f3-cbcadcbe535d req-c694f196-0275-4b40-b177-5649dea678b3 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:33.339273 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-aee4ce58-82dc-491e-a9f3-cbcadcbe535d req-c694f196-0275-4b40-b177-5649dea678b3 service nova] Creating event network-vif-plugged:7589db85-95c8-412c-a8f7-5db156ef7906 for instance eb66c484-73f0-441b-a575-f506a0caca8a on hv-cinder-maste Sep 25 01:05:33.345311 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-aee4ce58-82dc-491e-a9f3-cbcadcbe535d req-c694f196-0275-4b40-b177-5649dea678b3 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035288 Sep 25 01:05:33.345905 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 461/935] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:05:33 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:05:33.698182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:33.699008 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:33.706941 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:33.707502 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:33.707957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:33.708686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:33.709153 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:33.709613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:33.875352 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:33.884771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6b1164e3-ce1a-4d5b-96ee-e82ab6c9b8d3 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1575 microversion: 2.1 time: 0.193648 Sep 25 01:05:33.885395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 475/936] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:33 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1575 bytes in 195 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:34.914011 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:34.915352 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:34.926937 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:34.927832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:34.928650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:34.930007 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:34.930904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:34.932013 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:35.087901 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:35.096851 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ee417cb3-06c0-418a-8fa6-344d558a2cb2 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1562 microversion: 2.1 time: 0.189369 Sep 25 01:05:35.097484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 462/937] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:34 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1562 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:52.227394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:52.228007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:52.232782 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=13 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:52.236428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=48 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:52.242663 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:52.243165 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:52.243610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:52.244334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:52.244794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:52.245225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:52.624047 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lazy-loading 'services' on Instance uuid 1de127ae-8886-4d15-a6be-7b68902ccde8 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:05:52.723480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-445e9950-e102-436c-93d3-b40c0568fbbc tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/os-volume_attachments/e21ef985-5157-4b4a-ae45-1bd0ed2df8d7" status: 202 len: 0 microversion: 2.1 time: 0.501979 Sep 25 01:05:52.724479 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 476/938] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:05:52 2022] DELETE /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/os-volume_attachments/e21ef985-5157-4b4a-ae45-1bd0ed2df8d7 => generated 0 bytes in 504 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:05:53.899891 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d535a5f7-05ae-49f6-bb2a-39d4605fe2f1 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'action', calling method: >, body: {"os-stop": {}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:05:53.907672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=5 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:53.908972 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:05:53.917334 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d535a5f7-05ae-49f6-bb2a-39d4605fe2f1 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:53.918125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d535a5f7-05ae-49f6-bb2a-39d4605fe2f1 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:53.918940 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d535a5f7-05ae-49f6-bb2a-39d4605fe2f1 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:53.973064 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d535a5f7-05ae-49f6-bb2a-39d4605fe2f1 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Going to try to stop instance {{(pid=120450) force_stop /opt/stack/nova/nova/compute/api.py:2762}} Sep 25 01:05:54.058159 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d535a5f7-05ae-49f6-bb2a-39d4605fe2f1 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action" status: 202 len: 0 microversion: 2.1 time: 0.164224 Sep 25 01:05:54.059614 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 463/939] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:05:53 2022] POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action => generated 0 bytes in 168 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:05:54.075847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:54.079361 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:54.092156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:54.093556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:54.094370 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:54.095653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:54.096486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:54.097286 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:54.303130 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:54.314902 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c28ef5bc-2c50-4885-b634-b4bd932b21e5 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.248046 Sep 25 01:05:54.315905 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 477/940] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:54 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 251 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:55.351603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:55.352204 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:55.359976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:55.360614 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:55.361373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:55.362292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:55.363009 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:55.363651 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:55.671904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:55.686678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-028d4ffc-45d5-4a07-91a5-1527c08a7035 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.340359 Sep 25 01:05:55.688224 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 464/941] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:55 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 343 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:56.725040 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:56.725808 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:56.732840 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:56.734158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:56.735428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:56.735961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:56.736474 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:56.736951 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:56.892418 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:56.901441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-227f13a2-8bd8-4617-ba4d-7b9247f3f53d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.180207 Sep 25 01:05:56.902170 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 478/942] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:56 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:57.928416 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:57.928985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:57.931973 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 25 01:05:57.932158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Sun Sep 25 01:05:57 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 (ip 10.222.0.52) !!! Sep 25 01:05:57.941797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:57.942303 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:57.942745 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:57.943463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:57.943954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:57.944377 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:58.092104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:58.100482 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d6b383cd-8347-4f4d-b793-7654a761c31f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.176302 Sep 25 01:05:58.101310 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 465/943] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:57 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:05:59.126152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:05:59.126779 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:05:59.133911 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:59.134556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:59.135097 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:59.135939 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:05:59.136506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:05:59.137118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:05:59.306767 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:05:59.321675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3b335719-9669-4375-aa1f-7e0d3df24ddf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.199381 Sep 25 01:05:59.322743 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 479/944] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:05:59 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:00.363863 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:00.364490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:00.371040 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:00.371529 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:00.371964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:00.372670 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:00.373123 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:00.374061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:00.530631 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:00.539468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-448d9c7e-edd1-45f7-82ba-d5b5c7172e50 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.179813 Sep 25 01:06:00.539962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 466/945] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:00 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:01.563769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:01.564538 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:01.572103 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:01.572649 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:01.573122 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:01.573921 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:01.574420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:01.574966 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:01.736019 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:01.744978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b7c2f00b-6cef-4700-a37a-b419f084e337 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.185281 Sep 25 01:06:01.745626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 480/946] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:01 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:02.771813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:02.772724 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:02.781220 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:02.781655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:02.782093 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:02.783192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:02.783648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:02.784123 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:02.939811 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:02.948417 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6248b572-7986-47ad-9d56-e5e22a80f3de tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.181750 Sep 25 01:06:02.949002 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 467/947] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:02 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:03.978936 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:03.980001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:03.990975 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:03.992042 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:03.993245 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:03.995584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:03.996098 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:03.996538 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:04.162511 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:04.173110 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-64866bd7-9e2d-48e3-ba61-fb1913037856 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1675 microversion: 2.1 time: 0.203446 Sep 25 01:06:04.174102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 481/948] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:03 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1675 bytes in 206 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:05.199181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:05.200197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:05.208266 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:05.208909 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:05.209556 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:05.210372 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:05.210923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:05.211438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:05.376256 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:05.385515 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-343859fd-56de-4d17-9481-9dc52b64a36b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1652 microversion: 2.1 time: 0.190607 Sep 25 01:06:05.386125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 468/949] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:05 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1652 bytes in 192 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:05.409112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-990c9a66-e077-4119-b8e3-a220fcaf4bfe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'action', calling method: >, body: {"os-start": {}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:05.411940 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 25 01:06:05.412113 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Sun Sep 25 01:06:05 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action (ip 10.222.0.52) !!! Sep 25 01:06:05.422269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-990c9a66-e077-4119-b8e3-a220fcaf4bfe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:05.422895 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-990c9a66-e077-4119-b8e3-a220fcaf4bfe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:05.423466 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-990c9a66-e077-4119-b8e3-a220fcaf4bfe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:05.456007 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-990c9a66-e077-4119-b8e3-a220fcaf4bfe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Going to try to start instance {{(pid=120451) start /opt/stack/nova/nova/compute/api.py:2785}} Sep 25 01:06:05.513770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-990c9a66-e077-4119-b8e3-a220fcaf4bfe tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action" status: 202 len: 0 microversion: 2.1 time: 0.108347 Sep 25 01:06:05.515443 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 482/950] 10.222.0.52 () {60 vars in 1278 bytes} [Sun Sep 25 01:06:05 2022] POST /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/action => generated 0 bytes in 111 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 01:06:05.524402 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:05.525409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:05.534979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:05.535594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:05.536322 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:05.538521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:05.538521 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:05.539245 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:05.839133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:05.854723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-446f9df0-a119-4b15-832c-6968078dcba2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1661 microversion: 2.1 time: 0.334079 Sep 25 01:06:05.855770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 469/951] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:05 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1661 bytes in 336 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:06.887293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:06.888208 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:06.896835 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:06.897412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:06.897898 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:06.898674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:06.899234 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:06.899852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:07.349919 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:07.361722 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ecbe2837-28f4-4e4e-87e7-415663468937 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1661 microversion: 2.1 time: 0.480044 Sep 25 01:06:07.362483 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 483/952] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:06 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1661 bytes in 482 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:08.386841 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:08.387962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:08.397453 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:08.398081 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:08.398647 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:08.399872 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:08.400498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:08.401035 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:08.545799 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:08.548184 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:08.562229 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:08.563610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.003s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:08.564503 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:08.565976 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:08.567171 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:08.568286 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:08.625415 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:06:08.642021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:08.651986 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e0ae6da3-5bc1-4fac-8ce9-5f6e099fc4a2 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1665 microversion: 2.1 time: 0.269188 Sep 25 01:06:08.653729 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 470/953] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:08 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1665 bytes in 271 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:08.663297 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-9b9aeea2-7628-4a7f-bae6-047cf79486b5 req-2e29a41c-3404-4954-a389-01462106b31f service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "4814bc8c-42fd-4665-9123-2feba2f7887f", "tag": "9d97f0d8-3c7b-4ac3-8d16-6a80eec13764"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:08.686569 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-9b9aeea2-7628-4a7f-bae6-047cf79486b5 req-2e29a41c-3404-4954-a389-01462106b31f service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:08.688197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-9b9aeea2-7628-4a7f-bae6-047cf79486b5 req-2e29a41c-3404-4954-a389-01462106b31f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:08.689594 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-9b9aeea2-7628-4a7f-bae6-047cf79486b5 req-2e29a41c-3404-4954-a389-01462106b31f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:08.708693 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-9b9aeea2-7628-4a7f-bae6-047cf79486b5 req-2e29a41c-3404-4954-a389-01462106b31f service nova] Creating event network-changed:9d97f0d8-3c7b-4ac3-8d16-6a80eec13764 for instance 4814bc8c-42fd-4665-9123-2feba2f7887f on hv-cinder-maste Sep 25 01:06:08.717597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-9b9aeea2-7628-4a7f-bae6-047cf79486b5 req-2e29a41c-3404-4954-a389-01462106b31f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.060618 Sep 25 01:06:08.718888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 471/954] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:06:08 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 64 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:06:08.784182 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-af0c2fc2-8953-404d-869e-a410dac43cc4 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "DELETE /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 204 len: 0 microversion: 2.1 time: 0.246888 Sep 25 01:06:08.789700 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 484/955] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:06:08 2022] DELETE /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 0 bytes in 249 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 25 01:06:08.805317 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:08.805317 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:08.815668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:08.816259 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:08.816786 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:08.817988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:08.819618 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:08.820423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:09.013884 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:09.024983 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-24dfd946-3f57-4c7c-8855-224c5a624fa8 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1578 microversion: 2.1 time: 0.232604 Sep 25 01:06:09.025835 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 472/956] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:08 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1578 bytes in 235 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:10.055908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:10.057224 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:10.069247 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:10.070252 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:10.070441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:10.071196 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:10.071962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:10.072739 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:10.285580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:10.296914 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-21bc2a77-87d1-4f94-aa95-d94b2211ea52 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1578 microversion: 2.1 time: 0.248638 Sep 25 01:06:10.297914 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 485/957] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:10 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1578 bytes in 252 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:11.328872 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:11.330640 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:11.343999 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:11.345419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:11.346278 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:11.347455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:11.348214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:11.348909 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:11.429586 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-72811afa-9deb-4923-914a-644ea53759b2 req-d4bf17ac-6806-41d9-a4cf-177c45e97dc4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "4814bc8c-42fd-4665-9123-2feba2f7887f", "name": "network-vif-deleted", "tag": "9d97f0d8-3c7b-4ac3-8d16-6a80eec13764"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:11.462099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-72811afa-9deb-4923-914a-644ea53759b2 req-d4bf17ac-6806-41d9-a4cf-177c45e97dc4 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:11.463744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-72811afa-9deb-4923-914a-644ea53759b2 req-d4bf17ac-6806-41d9-a4cf-177c45e97dc4 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.004s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:11.464860 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-72811afa-9deb-4923-914a-644ea53759b2 req-d4bf17ac-6806-41d9-a4cf-177c45e97dc4 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:11.487847 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:11.503626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-72811afa-9deb-4923-914a-644ea53759b2 req-d4bf17ac-6806-41d9-a4cf-177c45e97dc4 service nova] Creating event network-vif-deleted:9d97f0d8-3c7b-4ac3-8d16-6a80eec13764 for instance 4814bc8c-42fd-4665-9123-2feba2f7887f on hv-cinder-maste Sep 25 01:06:11.509826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9f3798c2-a3f9-4471-abb1-8efb92df7c08 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 200 len: 1391 microversion: 2.1 time: 0.188523 Sep 25 01:06:11.511104 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 473/958] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:11 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 1391 bytes in 191 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:11.529496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-72811afa-9deb-4923-914a-644ea53759b2 req-d4bf17ac-6806-41d9-a4cf-177c45e97dc4 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.105316 Sep 25 01:06:11.531738 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 486/959] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:06:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:06:12.538954 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:12.539725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 4814bc8c-42fd-4665-9123-2feba2f7887f] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:12.547956 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:12.548461 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:12.548952 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:12.549816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:12.550366 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:12.550802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:12.568176 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] HTTP exception thrown: Instance 4814bc8c-42fd-4665-9123-2feba2f7887f could not be found. Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Request method failure captured: Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f HTTP/1.1 Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6d9c0dB9tCHvIml0BBsUSlOVo-qLb3eukyQgMLgvyRXH1CyMjUyaQc1f1JKazgmAE4J3bJO5T5Z_kKMpRncWkpX-mTfm_aKx9Cap4bu1LUkSDrvuHpNS_NhBfp3ENvyhM7JnJSpK88-o5SQC6xM3absHOF-r0IffzDxhAp_0Swmsps Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:06:12.569914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_46f19fd9ad1e4880961d1eee75883687", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}] Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: c19b10ebc31244dbb2b1539d94ac281d Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 4814bc8c-42fd-4665-9123-2feba2f7887f could not be found. Sep 25 01:06:12.573019 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '4814bc8c-42fd-4665-9123-2feba2f7887f'} Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 4814bc8c-42fd-4665-9123-2feba2f7887f could not be found. Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:06:12.574686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 4814bc8c-42fd-4665-9123-2feba2f7887f could not be found. Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 4814bc8c-42fd-4665-9123-2feba2f7887f could not be found. Sep 25 01:06:12.577643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:12.580173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Returning 404 to user: Instance 4814bc8c-42fd-4665-9123-2feba2f7887f could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:06:12.580173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2549e9ed-2d29-4f67-8ba8-a7904e9f84b5 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f" status: 404 len: 111 microversion: 2.1 time: 0.036931 Sep 25 01:06:12.580173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 474/960] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:12 2022] GET /compute/v2.1/servers/4814bc8c-42fd-4665-9123-2feba2f7887f => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:06:15.876940 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:15.878356 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:15.881481 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-8ea79c5d-884a-4db2-b708-3aa46f7020d9 req-82d7079d-5bf5-4185-9831-058ef11912c8 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "fdb14c73-6de8-4097-83d7-44f81a01abab", "tag": "b7527175-254d-4003-8922-2e8834ba4a91"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:15.890364 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:15.891598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:15.892574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:15.893864 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:15.894875 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:15.895615 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:15.899593 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-8ea79c5d-884a-4db2-b708-3aa46f7020d9 req-82d7079d-5bf5-4185-9831-058ef11912c8 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:15.900388 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-8ea79c5d-884a-4db2-b708-3aa46f7020d9 req-82d7079d-5bf5-4185-9831-058ef11912c8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:15.901502 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-8ea79c5d-884a-4db2-b708-3aa46f7020d9 req-82d7079d-5bf5-4185-9831-058ef11912c8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:15.924263 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-8ea79c5d-884a-4db2-b708-3aa46f7020d9 req-82d7079d-5bf5-4185-9831-058ef11912c8 service nova] Creating event network-changed:b7527175-254d-4003-8922-2e8834ba4a91 for instance fdb14c73-6de8-4097-83d7-44f81a01abab on hv-cinder-maste Sep 25 01:06:15.932949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-8ea79c5d-884a-4db2-b708-3aa46f7020d9 req-82d7079d-5bf5-4185-9831-058ef11912c8 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.057910 Sep 25 01:06:15.934397 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 475/961] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:06:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:06:15.948037 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:06:16.089795 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c7032f18-0e63-47d3-9655-f99cc585fcca tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "DELETE /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 204 len: 0 microversion: 2.1 time: 0.220355 Sep 25 01:06:16.094126 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 487/962] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:06:15 2022] DELETE /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 0 bytes in 224 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 25 01:06:16.114361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:16.115591 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:16.127112 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:16.128021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:16.128839 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:16.130199 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:16.131063 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:16.132024 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:16.369519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:16.381978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d89bf042-be69-42a2-b559-604096f281ce tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1577 microversion: 2.1 time: 0.284977 Sep 25 01:06:16.382816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 476/963] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:16 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1577 bytes in 288 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:17.411889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:17.413067 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:17.421532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:17.422427 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:17.423144 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:17.423984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:17.425244 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:17.425482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:17.430351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=93,nova_cell1:INSERT=4,nova_cell1:UPDATE=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:17.616739 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:17.626153 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2f13c2ae-351c-4433-aa31-2a1f8a0ff3ef tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1577 microversion: 2.1 time: 0.221085 Sep 25 01:06:17.626651 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 488/964] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:17 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1577 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:18.671094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:18.672675 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:18.679681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:18.680197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:18.680649 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:18.681460 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:18.681929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:18.682364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:18.925156 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:18.934471 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6c518662-169f-4adb-bd46-f9716faac9fc tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1459 microversion: 2.1 time: 0.268984 Sep 25 01:06:18.935061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 477/965] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:18 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1459 bytes in 270 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:19.058269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-69c44e66-71ca-43b7-abc5-6bd18473be8e req-2240ab16-be92-41cb-91ff-e57456eb4c2e service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "fdb14c73-6de8-4097-83d7-44f81a01abab", "name": "network-vif-deleted", "tag": "b7527175-254d-4003-8922-2e8834ba4a91"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:19.069807 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-69c44e66-71ca-43b7-abc5-6bd18473be8e req-2240ab16-be92-41cb-91ff-e57456eb4c2e service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:19.070338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-69c44e66-71ca-43b7-abc5-6bd18473be8e req-2240ab16-be92-41cb-91ff-e57456eb4c2e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:19.070714 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-69c44e66-71ca-43b7-abc5-6bd18473be8e req-2240ab16-be92-41cb-91ff-e57456eb4c2e service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:19.084730 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-69c44e66-71ca-43b7-abc5-6bd18473be8e req-2240ab16-be92-41cb-91ff-e57456eb4c2e service nova] Creating event network-vif-deleted:b7527175-254d-4003-8922-2e8834ba4a91 for instance fdb14c73-6de8-4097-83d7-44f81a01abab on hv-cinder-maste Sep 25 01:06:19.090777 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-69c44e66-71ca-43b7-abc5-6bd18473be8e req-2240ab16-be92-41cb-91ff-e57456eb4c2e service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035834 Sep 25 01:06:19.091819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 489/966] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:06:19 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 38 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:06:19.958761 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:19.959381 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:19.966846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:19.968200 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:19.968827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:19.969585 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:19.970049 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:19.970473 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:20.008831 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:20.009500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:20.009987 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:20.010485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lazy-loading 'fault' on Instance uuid fdb14c73-6de8-4097-83d7-44f81a01abab {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:06:20.067363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'c19b10ebc31244dbb2b1539d94ac281d', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f19fd9ad1e4880961d1eee75883687', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:20.080405 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a2a643f7-fdfd-4304-aa2f-4be0193ce97d tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 200 len: 1230 microversion: 2.1 time: 0.125473 Sep 25 01:06:20.081466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 478/967] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:19 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 1230 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:21.117441 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:21.118161 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: fdb14c73-6de8-4097-83d7-44f81a01abab] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:21.124835 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:21.125374 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:21.125817 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:21.126532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:21.126989 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:21.127415 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:21.142424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] HTTP exception thrown: Instance fdb14c73-6de8-4097-83d7-44f81a01abab could not be found. Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Request method failure captured: Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab HTTP/1.1 Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6d9c0dB9tCHvIml0BBsUSlOVo-qLb3eukyQgMLgvyRXH1CyMjUyaQc1f1JKazgmAE4J3bJO5T5Z_kKMpRncWkpX-mTfm_aKx9Cap4bu1LUkSDrvuHpNS_NhBfp3ENvyhM7JnJSpK88-o5SQC6xM3absHOF-r0IffzDxhAp_0Swmsps Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:06:21.144101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_46f19fd9ad1e4880961d1eee75883687", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}] Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: c19b10ebc31244dbb2b1539d94ac281d Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance fdb14c73-6de8-4097-83d7-44f81a01abab could not be found. Sep 25 01:06:21.145702 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': 'fdb14c73-6de8-4097-83d7-44f81a01abab'} Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance fdb14c73-6de8-4097-83d7-44f81a01abab could not be found. Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:06:21.146644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance fdb14c73-6de8-4097-83d7-44f81a01abab could not be found. Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance fdb14c73-6de8-4097-83d7-44f81a01abab could not be found. Sep 25 01:06:21.148167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.149619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Returning 404 to user: Instance fdb14c73-6de8-4097-83d7-44f81a01abab could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:06:21.149619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-2dced2f5-5447-410b-993c-44950d713d23 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab" status: 404 len: 111 microversion: 2.1 time: 0.032989 Sep 25 01:06:21.149619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 490/968] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:21 2022] GET /compute/v2.1/servers/fdb14c73-6de8-4097-83d7-44f81a01abab => generated 111 bytes in 34 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:06:21.782894 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:21.783528 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:21.790709 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:21.791210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:21.791657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:21.792389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:21.792849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:21.793311 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:21.809167 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] HTTP exception thrown: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Request method failure captured: Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 HTTP/1.1 Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6d9c0dB9tCHvIml0BBsUSlOVo-qLb3eukyQgMLgvyRXH1CyMjUyaQc1f1JKazgmAE4J3bJO5T5Z_kKMpRncWkpX-mTfm_aKx9Cap4bu1LUkSDrvuHpNS_NhBfp3ENvyhM7JnJSpK88-o5SQC6xM3absHOF-r0IffzDxhAp_0Swmsps Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:06:21.811080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_46f19fd9ad1e4880961d1eee75883687", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}] Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: c19b10ebc31244dbb2b1539d94ac281d Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.814061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '63e5af09-dbe0-471f-a5bd-9519f11c8d70'} Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:06:21.815823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.818574 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.821519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Returning 404 to user: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:06:21.821519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5352fa7c-c911-4a48-8c2c-d832a19db464 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "DELETE /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 404 len: 111 microversion: 2.1 time: 0.034635 Sep 25 01:06:21.821519 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 479/969] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:06:21 2022] DELETE /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:06:21.822014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:21.822616 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] [instance: 63e5af09-dbe0-471f-a5bd-9519f11c8d70] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:21.830036 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:21.830539 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:21.830988 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:21.831717 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:21.832179 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:21.832611 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:21.848100 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] HTTP exception thrown: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Request method failure captured: Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 HTTP/1.1 Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6d9c0dB9tCHvIml0BBsUSlOVo-qLb3eukyQgMLgvyRXH1CyMjUyaQc1f1JKazgmAE4J3bJO5T5Z_kKMpRncWkpX-mTfm_aKx9Cap4bu1LUkSDrvuHpNS_NhBfp3ENvyhM7JnJSpK88-o5SQC6xM3absHOF-r0IffzDxhAp_0Swmsps Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:06:21.849639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_46f19fd9ad1e4880961d1eee75883687", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f19fd9ad1e4880961d1eee75883687"}]}] Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 46f19fd9ad1e4880961d1eee75883687 Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-TestVolumeBootPattern-1642664612 Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: c19b10ebc31244dbb2b1539d94ac281d Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-TestVolumeBootPattern-1642664612-project Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.852215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '63e5af09-dbe0-471f-a5bd-9519f11c8d70'} Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:06:21.853580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. Sep 25 01:06:21.855811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:06:21.858235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Returning 404 to user: Instance 63e5af09-dbe0-471f-a5bd-9519f11c8d70 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:06:21.858235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-719145e6-29cb-4ed2-985d-e1b4e2bde8c2 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70" status: 404 len: 111 microversion: 2.1 time: 0.033095 Sep 25 01:06:21.858235 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 491/970] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:21 2022] GET /compute/v2.1/servers/63e5af09-dbe0-471f-a5bd-9519f11c8d70 => generated 111 bytes in 38 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:06:23.238653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-63f2dddc-225c-4a8e-ab2d-591d3fd92940 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] Calling method '.version_select at 0x7fe2311f88b0>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:23.263819 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-63f2dddc-225c-4a8e-ab2d-591d3fd92940 tempest-TestVolumeBootPattern-1642664612 tempest-TestVolumeBootPattern-1642664612-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1527375203" status: 202 len: 0 microversion: 2.1 time: 0.029077 Sep 25 01:06:23.265427 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 480/971] 10.222.0.52 () {58 vars in 1263 bytes} [Sun Sep 25 01:06:23 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestVolumeBootPattern-1527375203 => generated 0 bytes in 31 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 01:06:42.114584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:42.117060 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:42.119731 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=42,nova_api:UPDATE=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:42.120438 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=10 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:42.128985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:42.129624 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:42.130347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:42.131114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:42.131614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:42.132083 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:42.168883 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-0ae8c2aa-ac16-4b4c-87c6-eb6057ebc974 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.059148 Sep 25 01:06:42.169607 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 492/972] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:06:42 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8/os-volume_attachments => generated 25 bytes in 61 msecs (HTTP/1.1 200) 9 headers in 356 bytes (1 switches on core 0) Sep 25 01:06:42.240980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=92,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:42.241758 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=39,nova_api:DELETE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:06:42.246426 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:42.247815 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:42.257767 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:42.258261 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:42.258699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:42.259403 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:42.259859 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:42.260285 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:42.292775 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:06:42.408597 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2a91599c-4f81-4eff-8dbd-53fbb73a8525 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 204 len: 0 microversion: 2.1 time: 0.168297 Sep 25 01:06:42.409729 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 481/973] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:06:42 2022] DELETE /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:06:42.422469 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:42.423579 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:42.434251 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:42.435138 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:42.437058 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:42.438415 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:42.439274 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:42.440066 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:42.604208 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:42.613681 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c12f3f6a-eacd-471c-92fd-5ce29380190b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1671 microversion: 2.1 time: 0.198643 Sep 25 01:06:42.614277 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 493/974] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:42 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1671 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:43.641168 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:43.641979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:43.649156 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:43.649727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:43.650195 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:43.650928 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:43.651386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:43.651806 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:43.809825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:43.819423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6bfc2e2e-b1aa-45d4-a6a6-d1ac05436af6 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1671 microversion: 2.1 time: 0.183735 Sep 25 01:06:43.820083 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 482/975] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:43 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1671 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:44.849187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:44.850447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:44.863035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:44.864020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:44.864794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:44.866077 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:44.866933 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:44.867691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:45.074416 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:45.085217 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5c01bd31-b448-4cf7-9ed7-a9385407cf3b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1671 microversion: 2.1 time: 0.244044 Sep 25 01:06:45.086310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 494/976] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:44 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1671 bytes in 246 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:46.136594 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:46.137457 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:46.158647 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:46.159338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:46.159889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:46.160823 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:46.161763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:46.162766 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:46.244054 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:46.244859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:46.245672 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:46.246735 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lazy-loading 'fault' on Instance uuid 1de127ae-8886-4d15-a6be-7b68902ccde8 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:06:46.249087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-e974aff1-9766-445c-bd29-7e2421a915e1 req-040be0ca-0420-47df-958c-9da9fe6f5f02 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1de127ae-8886-4d15-a6be-7b68902ccde8", "name": "network-vif-deleted", "tag": "c71c9659-825c-4f3d-b945-2c964a1f67a7"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:46.269945 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e974aff1-9766-445c-bd29-7e2421a915e1 req-040be0ca-0420-47df-958c-9da9fe6f5f02 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:46.271211 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e974aff1-9766-445c-bd29-7e2421a915e1 req-040be0ca-0420-47df-958c-9da9fe6f5f02 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:46.272253 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e974aff1-9766-445c-bd29-7e2421a915e1 req-040be0ca-0420-47df-958c-9da9fe6f5f02 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:46.296572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-e974aff1-9766-445c-bd29-7e2421a915e1 req-040be0ca-0420-47df-958c-9da9fe6f5f02 service nova] Creating event network-vif-deleted:c71c9659-825c-4f3d-b945-2c964a1f67a7 for instance 1de127ae-8886-4d15-a6be-7b68902ccde8 on hv-cinder-maste Sep 25 01:06:46.306027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-e974aff1-9766-445c-bd29-7e2421a915e1 req-040be0ca-0420-47df-958c-9da9fe6f5f02 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.431324 Sep 25 01:06:46.307466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 483/977] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:06:45 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 435 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:06:46.323598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:46.337343 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3adcfb47-0ebe-41be-ae83-b37cc67031ee tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 200 len: 1326 microversion: 2.1 time: 0.208919 Sep 25 01:06:46.341316 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 495/978] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:46 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 1326 bytes in 212 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:47.382181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:47.383899 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:47.396570 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:47.397542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:47.398382 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:47.399652 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:47.400995 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:47.401861 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:47.430762 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] HTTP exception thrown: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Request method failure captured: Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 HTTP/1.1 Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6jzQ8ZGzZbsRNfl9mXkdLfXWHvwRSpA0SXP7A_oXBAYMbdKJOjlYvr8B_BfwcdUtJJN8BI8SPqL-aMgg5q4U3v5SVXcxzP40NPeDSdsurJSB5u579RBlZFTKMs4VmEMmE5E9fooDCVVv5dB6TpRCVhQnau9VkVTaJagbdgobkhvLXc Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:06:47.433903 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f37b53a21b19482b8dcac6c38505a50b", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}] Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 1e515c9c8dcf4a1aa7dd74fb7655adf4 Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:06:47.437389 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '1de127ae-8886-4d15-a6be-7b68902ccde8'} Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:06:47.440392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:06:47.443289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:47.445790 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Returning 404 to user: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:06:47.445790 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-bae0b583-bc48-4dbb-8407-8b3708dde257 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 404 len: 111 microversion: 2.1 time: 0.063578 Sep 25 01:06:47.445790 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 484/979] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:47 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 111 bytes in 66 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:06:47.452925 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3351b822-664c-41db-8ccf-69958b3c97ff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '.version_select at 0x7fe22d7fec10>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:47.486978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3351b822-664c-41db-8ccf-69958b3c97ff tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-31100642" status: 202 len: 0 microversion: 2.1 time: 0.041864 Sep 25 01:06:47.487979 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 496/980] 10.222.0.52 () {58 vars in 1215 bytes} [Sun Sep 25 01:06:47 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-31100642 => generated 0 bytes in 44 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:06:48.257514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2f83a14e-b54f-4174-bcdb-ef42d8e4ea7c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'create', calling method: .version_select at 0x7fe22d7964c0>, body: {"keypair": {"name": "tempest-keypair-1546838375", "public_key": "ecdsa-sha2-nistp384 AAAAE2VjZHNhLXNoYTItbmlzdHAzODQAAAAIbmlzdHAzODQAAABhBBDXjJYiaOYd8Q3ADbTIOsgGIgpDqoqbALfIYEohgcQuJOLlhTklHCtpDVG3VPucIS5EC8fcErp2EeJfId+OoSYulnb09X5Gn72QWR7GzaeSfjItvPrdwg7lWGFGHYNxCQ=="}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:48.275875 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-2f83a14e-b54f-4174-bcdb-ef42d8e4ea7c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Getting quotas for project f37b53a21b19482b8dcac6c38505a50b. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:06:48.281085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-2f83a14e-b54f-4174-bcdb-ef42d8e4ea7c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Getting quotas for user 1e515c9c8dcf4a1aa7dd74fb7655adf4 and project f37b53a21b19482b8dcac6c38505a50b. Resources: {'key_pairs'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:06:48.307996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2f83a14e-b54f-4174-bcdb-ef42d8e4ea7c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/os-keypairs" status: 200 len: 386 microversion: 2.1 time: 0.054436 Sep 25 01:06:48.308606 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 485/981] 10.222.0.52 () {60 vars in 1159 bytes} [Sun Sep 25 01:06:48 2022] POST /compute/v2.1/os-keypairs => generated 386 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:06:50.232473 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-AttachVolumeTestJSON-server-1085929731", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "adminPass": "***", "networks": [{"uuid": "dbea2e08-a361-48c8-95fc-03b3fc326d51"}], "security_groups": [{"name": "tempest-securitygroup--1323704980"}], "key_name": "tempest-keypair-1546838375", "user_data": "}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:50.371522 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] validate_networks() for [('dbea2e08-a361-48c8-95fc-03b3fc326d51', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:06:51.107750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:06:51.108470 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:06:51.109359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:06:51.109958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:06:51.110423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:06:51.121501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:51.122146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:51.122689 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:51.124008 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:51.124475 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:51.124902 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:51.146341 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Getting quotas for project f37b53a21b19482b8dcac6c38505a50b. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:06:51.151206 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Getting quotas for user 1e515c9c8dcf4a1aa7dd74fb7655adf4 and project f37b53a21b19482b8dcac6c38505a50b. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:06:51.175821 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:06:51.179034 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:06:51.235414 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-38d28687-e2ec-4ad2-840d-6e245e13394d tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 402 microversion: 2.1 time: 1.007881 Sep 25 01:06:51.236632 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 497/982] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:06:50 2022] POST /compute/v2.1/servers => generated 402 bytes in 1010 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 01:06:51.251834 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ada05cfe-ca16-42c7-b402-2e5871bbf7e0 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:51.252913 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ada05cfe-ca16-42c7-b402-2e5871bbf7e0 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:51.331733 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ada05cfe-ca16-42c7-b402-2e5871bbf7e0 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:51.340738 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ada05cfe-ca16-42c7-b402-2e5871bbf7e0 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1238 microversion: 2.1 time: 0.096267 Sep 25 01:06:51.341840 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 486/983] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:51 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1238 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:52.368034 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:52.368673 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:52.379463 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:52.380287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:52.381146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:52.382524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:52.383226 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:52.383983 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:52.467938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:52.478021 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-400f33e1-4306-4a0e-afc6-e0dfbc0ba598 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1290 microversion: 2.1 time: 0.115985 Sep 25 01:06:52.478577 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 498/984] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:52 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1290 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:53.513169 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:53.514367 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:53.526551 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:53.527429 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:53.528232 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:53.529581 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:53.530450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:53.531212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:53.628895 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:53.629728 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:53.643064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:53.643919 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:53.644691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:53.645889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:53.646471 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:53.647157 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:53.663530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:53.673512 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-12b853c4-5646-41c2-8c89-c8e6cf66a941 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1296 microversion: 2.1 time: 0.166222 Sep 25 01:06:53.674776 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 487/985] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:53 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1296 bytes in 168 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:53.683296 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-b7d9632b-dd5a-4722-ab1a-3204324e2b8c req-a102d8d5-2272-4634-8d22-7d45fd732c6a service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "eb66c484-73f0-441b-a575-f506a0caca8a", "tag": "7589db85-95c8-412c-a8f7-5db156ef7906"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:53.706461 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:06:53.707306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b7d9632b-dd5a-4722-ab1a-3204324e2b8c req-a102d8d5-2272-4634-8d22-7d45fd732c6a service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:53.707875 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b7d9632b-dd5a-4722-ab1a-3204324e2b8c req-a102d8d5-2272-4634-8d22-7d45fd732c6a service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:53.708392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b7d9632b-dd5a-4722-ab1a-3204324e2b8c req-a102d8d5-2272-4634-8d22-7d45fd732c6a service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:53.736258 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-b7d9632b-dd5a-4722-ab1a-3204324e2b8c req-a102d8d5-2272-4634-8d22-7d45fd732c6a service nova] Creating event network-changed:7589db85-95c8-412c-a8f7-5db156ef7906 for instance eb66c484-73f0-441b-a575-f506a0caca8a on hv-cinder-maste Sep 25 01:06:53.746749 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-b7d9632b-dd5a-4722-ab1a-3204324e2b8c req-a102d8d5-2272-4634-8d22-7d45fd732c6a service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.070462 Sep 25 01:06:53.747463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 488/986] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:06:53 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:06:53.860639 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-536683ec-4553-4d06-a061-2558281e637b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "DELETE /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 204 len: 0 microversion: 2.1 time: 0.236277 Sep 25 01:06:53.863758 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 499/987] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:06:53 2022] DELETE /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 0 bytes in 244 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:06:53.876073 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:53.877056 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:53.885114 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:53.886266 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:53.887139 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:53.888573 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:53.889782 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:53.890702 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:54.350466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:54.362967 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7d5d7018-21f4-44c3-8f01-122c3166055d tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.493987 Sep 25 01:06:54.364096 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 489/988] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:53 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 496 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:54.705885 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:54.707068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:54.720664 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:54.721142 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:54.721945 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.002s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:54.723080 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:54.724287 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:54.724877 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:54.945176 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:54.954719 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-b7997ecc-a124-4633-ba02-bf8ff4020a69 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1364 microversion: 2.1 time: 0.258687 Sep 25 01:06:54.972329 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 500/989] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:54 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1364 bytes in 277 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:55.405494 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:55.406213 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:55.413621 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:55.414134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:55.414669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:55.415393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:55.415929 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:55.416361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:55.905398 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:55.915219 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-2ace87e6-ee40-43a6-901f-ad63883ea88b tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1568 microversion: 2.1 time: 0.514762 Sep 25 01:06:55.915693 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 490/990] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:55 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1568 bytes in 516 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:55.980106 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:55.981156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:55.989227 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:55.989961 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:55.990550 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:55.991386 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:55.992054 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:55.992593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:56.377740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:56.392771 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e42f1d2d-8f17-4f0d-8756-ea5cb8a6f92b tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1364 microversion: 2.1 time: 0.416435 Sep 25 01:06:56.409914 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 501/991] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:55 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1364 bytes in 434 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:56.848504 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-fbd62989-cc23-41a5-bc9d-cc9a04a4b1f5 req-9b67911c-4090-458f-ac85-9b37ed85e109 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a098f4d1-9c50-4e95-a5aa-e7cb07850ad4", "tag": "1efe19b6-e801-45ed-8aac-c94c3a3e0758"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:56.885636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-fbd62989-cc23-41a5-bc9d-cc9a04a4b1f5 req-9b67911c-4090-458f-ac85-9b37ed85e109 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:56.886983 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-fbd62989-cc23-41a5-bc9d-cc9a04a4b1f5 req-9b67911c-4090-458f-ac85-9b37ed85e109 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:56.887861 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-fbd62989-cc23-41a5-bc9d-cc9a04a4b1f5 req-9b67911c-4090-458f-ac85-9b37ed85e109 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:56.910799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-fbd62989-cc23-41a5-bc9d-cc9a04a4b1f5 req-9b67911c-4090-458f-ac85-9b37ed85e109 service nova] Creating event network-changed:1efe19b6-e801-45ed-8aac-c94c3a3e0758 for instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 on hv-cinder-maste Sep 25 01:06:56.921386 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-fbd62989-cc23-41a5-bc9d-cc9a04a4b1f5 req-9b67911c-4090-458f-ac85-9b37ed85e109 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.095744 Sep 25 01:06:56.924006 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 491/992] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:06:56 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 103 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:06:56.966773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:56.972198 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:56.980882 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:56.981420 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:56.982156 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:56.982905 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:56.983523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:56.983957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:57.018701 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-d0826a0d-2f49-47ce-b9f8-4797f7b70668 req-2fa41287-a2af-42e5-b892-eee200818bb8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "eb66c484-73f0-441b-a575-f506a0caca8a", "name": "network-vif-deleted", "tag": "7589db85-95c8-412c-a8f7-5db156ef7906"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:06:57.031281 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d0826a0d-2f49-47ce-b9f8-4797f7b70668 req-2fa41287-a2af-42e5-b892-eee200818bb8 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:57.031768 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d0826a0d-2f49-47ce-b9f8-4797f7b70668 req-2fa41287-a2af-42e5-b892-eee200818bb8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:57.032209 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d0826a0d-2f49-47ce-b9f8-4797f7b70668 req-2fa41287-a2af-42e5-b892-eee200818bb8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:57.045086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-d0826a0d-2f49-47ce-b9f8-4797f7b70668 req-2fa41287-a2af-42e5-b892-eee200818bb8 service nova] Creating event network-vif-deleted:7589db85-95c8-412c-a8f7-5db156ef7906 for instance eb66c484-73f0-441b-a575-f506a0caca8a on hv-cinder-maste Sep 25 01:06:57.050861 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-d0826a0d-2f49-47ce-b9f8-4797f7b70668 req-2fa41287-a2af-42e5-b892-eee200818bb8 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.121097 Sep 25 01:06:57.051471 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 502/993] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:06:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 123 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:06:57.077603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:57.090449 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-61538efd-d6aa-4042-a862-423a30c31caa tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1381 microversion: 2.1 time: 0.133703 Sep 25 01:06:57.097214 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 492/994] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:56 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1381 bytes in 137 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:57.431724 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:57.432890 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:57.441571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:57.442211 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:57.442660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:57.443578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:57.444282 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:57.444708 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:57.619741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:57.628994 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5d96639c-05c9-4bc8-8788-6d2d8425ccec tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1364 microversion: 2.1 time: 0.201590 Sep 25 01:06:57.629640 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 503/995] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:57 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1364 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:58.116550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:58.117158 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:58.124051 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:58.124550 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:58.124993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:58.125751 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:58.126208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:58.126636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:58.161586 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:58.162082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:58.162524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:58.162979 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lazy-loading 'fault' on Instance uuid eb66c484-73f0-441b-a575-f506a0caca8a {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:06:58.209688 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '598ec83f7b7f4a9f90543218852aaf14', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '69a3d6ce251e480d866a86ed9f6f2eb9', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:58.219311 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7f0d3568-22dc-409f-a165-65123525c070 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 200 len: 1223 microversion: 2.1 time: 0.106670 Sep 25 01:06:58.219905 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 493/996] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:58 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 1223 bytes in 108 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:06:58.653017 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:58.654146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:58.662395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:58.663064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:58.663609 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:58.664480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:58.665042 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:58.665613 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:58.821442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:06:58.837532 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5edb998c-f739-4f78-a74b-9c2b38b215c8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1364 microversion: 2.1 time: 0.188237 Sep 25 01:06:58.849288 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 504/997] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:58 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1364 bytes in 190 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:06:59.244257 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:59.245449 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] [instance: eb66c484-73f0-441b-a575-f506a0caca8a] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:59.254438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:59.255172 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:59.255672 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:59.256432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:59.256931 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:59.257443 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:59.273779 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] HTTP exception thrown: Instance eb66c484-73f0-441b-a575-f506a0caca8a could not be found. Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Request method failure captured: Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a HTTP/1.1 Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6iCk2Td2jk9d4Bk_lQN8C8KNUJuikHve0oulZcBvN-Id8UNbH62EgRkr0rjCnODIgT1kuDfcRac_K_2oKwMsEeAd9J8AU5vbU6m_cOuaYHHEgKRz-jERF4BV5ivUX4fekHHzq5D0RaYIzQhZmp51FPrLIOp-uCIgwy8Qz-wWIbasdc Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 69a3d6ce251e480d866a86ed9f6f2eb9 Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-TestShelveInstance-384141591 Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:06:59.275714 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_69a3d6ce251e480d866a86ed9f6f2eb9", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/69a3d6ce251e480d866a86ed9f6f2eb9"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/69a3d6ce251e480d866a86ed9f6f2eb9"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/69a3d6ce251e480d866a86ed9f6f2eb9"}]}] Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-TestShelveInstance-384141591 Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 69a3d6ce251e480d866a86ed9f6f2eb9 Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-TestShelveInstance-384141591 Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-TestShelveInstance-384141591-project Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 598ec83f7b7f4a9f90543218852aaf14 Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-TestShelveInstance-384141591-project Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance eb66c484-73f0-441b-a575-f506a0caca8a could not be found. Sep 25 01:06:59.278293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'eb66c484-73f0-441b-a575-f506a0caca8a'} Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance eb66c484-73f0-441b-a575-f506a0caca8a could not be found. Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:06:59.281531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance eb66c484-73f0-441b-a575-f506a0caca8a could not be found. Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance eb66c484-73f0-441b-a575-f506a0caca8a could not be found. Sep 25 01:06:59.283959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:06:59.286146 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Returning 404 to user: Instance eb66c484-73f0-441b-a575-f506a0caca8a could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:06:59.286146 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b6cc3d0d-9667-4889-a506-dfaba2ccbdb6 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a" status: 404 len: 111 microversion: 2.1 time: 0.038013 Sep 25 01:06:59.286146 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 494/998] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:59 2022] GET /compute/v2.1/servers/eb66c484-73f0-441b-a575-f506a0caca8a => generated 111 bytes in 40 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:06:59.478724 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-18bd9d51-4e3b-44e0-bcce-afa8fc29b777 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] Calling method '.version_select at 0x7fe22da78820>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:59.508046 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-18bd9d51-4e3b-44e0-bcce-afa8fc29b777 tempest-TestShelveInstance-384141591 tempest-TestShelveInstance-384141591-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-102314433" status: 202 len: 0 microversion: 2.1 time: 0.036610 Sep 25 01:06:59.508812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 505/999] 10.222.0.52 () {58 vars in 1251 bytes} [Sun Sep 25 01:06:59 2022] DELETE /compute/v2.1/os-keypairs/tempest-TestShelveInstance-102314433 => generated 0 bytes in 39 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:06:59.869565 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:06:59.870498 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:06:59.879483 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:59.880821 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:59.881719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:06:59.882705 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:06:59.883634 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:06:59.884494 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:00.137054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:07:00.145992 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e3993e55-f369-47c7-a881-d461019aa3ea tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1364 microversion: 2.1 time: 0.286043 Sep 25 01:07:00.146658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 495/1000] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:06:59 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1364 bytes in 287 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:07:01.167751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:01.172934 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:07:01.173618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:01.175646 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=6 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:01.184259 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:01.184881 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:01.185453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:01.186250 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:01.186628 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:01.187068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:01.417456 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:07:01.426881 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-bd2f9f22-a2ef-47ac-9f94-18b70cb8a578 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1364 microversion: 2.1 time: 0.260652 Sep 25 01:07:01.427583 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 506/1001] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:07:01 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1364 bytes in 262 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:07:02.066244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=6 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:02.144955 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-f10b1a2e-9688-4f27-b1fc-4bba2baaa37f req-fd2d68fd-d092-4471-9685-8034cec50d69 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a098f4d1-9c50-4e95-a5aa-e7cb07850ad4", "name": "network-vif-plugged", "status": "completed", "tag": "1efe19b6-e801-45ed-8aac-c94c3a3e0758"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:07:02.156430 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-f10b1a2e-9688-4f27-b1fc-4bba2baaa37f req-fd2d68fd-d092-4471-9685-8034cec50d69 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:02.157081 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-f10b1a2e-9688-4f27-b1fc-4bba2baaa37f req-fd2d68fd-d092-4471-9685-8034cec50d69 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:02.157681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-f10b1a2e-9688-4f27-b1fc-4bba2baaa37f req-fd2d68fd-d092-4471-9685-8034cec50d69 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:02.175645 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-f10b1a2e-9688-4f27-b1fc-4bba2baaa37f req-fd2d68fd-d092-4471-9685-8034cec50d69 service nova] Creating event network-vif-plugged:1efe19b6-e801-45ed-8aac-c94c3a3e0758 for instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 on hv-cinder-maste Sep 25 01:07:02.182216 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-f10b1a2e-9688-4f27-b1fc-4bba2baaa37f req-fd2d68fd-d092-4471-9685-8034cec50d69 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.128458 Sep 25 01:07:02.183488 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 496/1002] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:07:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:07:02.468020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:07:02.468880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:02.476606 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:02.484429 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:02.484999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.008s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:02.485825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:02.486574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:02.487314 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:02.872113 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:07:02.885307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d0c4c40e-4c14-4a18-80df-cd3e20621a65 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1364 microversion: 2.1 time: 0.422146 Sep 25 01:07:02.886159 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 507/1003] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:07:02 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1364 bytes in 424 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:07:03.916603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:07:03.917618 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:03.933943 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:03.935428 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:03.936446 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:03.938005 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:03.939049 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:03.939900 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:04.210582 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:07:04.226515 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-965c1658-a65a-464a-9b76-d10424fb6f31 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1364 microversion: 2.1 time: 0.318076 Sep 25 01:07:04.227402 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 497/1004] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:07:03 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1364 bytes in 321 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:07:05.258545 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:07:05.259350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:05.269879 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:05.270697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:05.271486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:05.272239 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:05.272763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:05.273236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:05.659998 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:07:05.669336 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-005d780f-c03e-4021-8780-66b4a697fc8c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1550 microversion: 2.1 time: 0.417187 Sep 25 01:07:05.670236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 508/1005] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:07:05 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1550 bytes in 419 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:07:05.694466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:07:05.695116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:05.702966 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:05.703466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:05.703974 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:05.704719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:05.705186 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:05.705650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:05.795699 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-05b23c5b-6422-4ac5-933b-331addc47481 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-interface" status: 200 len: 285 microversion: 2.1 time: 0.105309 Sep 25 01:07:05.796416 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 498/1006] 10.222.0.52 () {58 vars in 1275 bytes} [Sun Sep 25 01:07:05 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-interface => generated 285 bytes in 107 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:07:07.469734 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-d3619a19-66a9-47ba-9e0f-323dc22fbfe3 req-563efa12-83e0-4b3e-b47a-bdcac8de43ce service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "a098f4d1-9c50-4e95-a5aa-e7cb07850ad4", "tag": "1efe19b6-e801-45ed-8aac-c94c3a3e0758"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:07:07.492327 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d3619a19-66a9-47ba-9e0f-323dc22fbfe3 req-563efa12-83e0-4b3e-b47a-bdcac8de43ce service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:07.493376 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d3619a19-66a9-47ba-9e0f-323dc22fbfe3 req-563efa12-83e0-4b3e-b47a-bdcac8de43ce service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:07.494248 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-d3619a19-66a9-47ba-9e0f-323dc22fbfe3 req-563efa12-83e0-4b3e-b47a-bdcac8de43ce service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:07.519828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-d3619a19-66a9-47ba-9e0f-323dc22fbfe3 req-563efa12-83e0-4b3e-b47a-bdcac8de43ce service nova] Creating event network-changed:1efe19b6-e801-45ed-8aac-c94c3a3e0758 for instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 on hv-cinder-maste Sep 25 01:07:07.526269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-d3619a19-66a9-47ba-9e0f-323dc22fbfe3 req-563efa12-83e0-4b3e-b47a-bdcac8de43ce service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.066776 Sep 25 01:07:07.527112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 509/1007] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:07:07 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 69 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:07:42.263693 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=54,nova_api:UPDATE=1,nova_api:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:42.264798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=80,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:42.270364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:07:42.270917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:42.279924 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:42.280425 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:42.281075 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:42.281854 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:42.282317 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:42.282749 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:42.311524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a142daa9-75d6-4843-af1a-ec7f5ce41c0f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/ips" status: 200 len: 145 microversion: 2.1 time: 0.050596 Sep 25 01:07:42.312125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 499/1008] 10.222.0.52 () {58 vars in 1248 bytes} [Sun Sep 25 01:07:42 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/ips => generated 145 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:07:43.778679 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=58,nova_api:DELETE=2,nova_api:INSERT=3,nova_api:UPDATE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:43.779576 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=94,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:07:43.784396 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1b5834f9-c7ed-4147-82f7-fe28d96f9928"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:07:43.786971 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:43.799056 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:43.799678 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:43.800177 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:43.800952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:43.801551 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:43.802070 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:44.131445 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-1c42bec4-6e79-4f7d-a93b-2792765a7f73 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.354683 Sep 25 01:07:44.132395 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 510/1009] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 01:07:43 2022] POST /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments => generated 194 bytes in 357 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:07:51.860455 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:07:51.863212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:51.871305 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:51.872201 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:51.872744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:51.873610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:51.874078 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:51.874513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:51.911134 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-771eee7c-a355-4d08-b470-bca581b1920c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments" status: 200 len: 197 microversion: 2.1 time: 0.054918 Sep 25 01:07:51.911754 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 500/1010] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:07:51 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments => generated 197 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:07:51.929549 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:07:51.930525 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:51.941333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:51.942013 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:51.942611 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:51.943619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:51.944263 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:51.944852 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:51.980761 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c954a338-db12-4e7d-9fa2-52e8d1298419 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1b5834f9-c7ed-4147-82f7-fe28d96f9928" status: 200 len: 194 microversion: 2.1 time: 0.059416 Sep 25 01:07:51.982072 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 511/1011] 10.222.0.52 () {58 vars in 1413 bytes} [Sun Sep 25 01:07:51 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1b5834f9-c7ed-4147-82f7-fe28d96f9928 => generated 194 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:07:53.436771 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "1607a9b6-f846-4a40-ba45-33a91b4898c1"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:07:53.439323 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:07:53.446461 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:53.446962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:53.447407 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:53.448145 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:07:53.448604 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:07:53.449044 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:07:53.780171 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d83a6879-e638-41a4-87f0-921f028134aa tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "POST /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.347588 Sep 25 01:07:53.780882 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 501/1012] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 01:07:53 2022] POST /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments => generated 194 bytes in 349 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:08:01.524038 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:01.526491 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:01.535027 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:01.535535 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:01.535980 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:01.536814 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:01.537289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:01.537781 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:01.575273 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-55c2f795-3906-4393-a567-f5bf72fe5ad4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments" status: 200 len: 371 microversion: 2.1 time: 0.054926 Sep 25 01:08:01.575999 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 512/1013] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:08:01 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments => generated 371 bytes in 57 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:08:01.748175 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:01.748827 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:01.755636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:01.756135 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:01.756573 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:01.757329 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:01.757789 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:01.758207 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:01.792838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9f9f3fa9-7c75-4d6f-9f25-8413796da783 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1b5834f9-c7ed-4147-82f7-fe28d96f9928" status: 200 len: 194 microversion: 2.1 time: 0.050385 Sep 25 01:08:01.793526 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 502/1014] 10.222.0.52 () {58 vars in 1413 bytes} [Sun Sep 25 01:08:01 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1b5834f9-c7ed-4147-82f7-fe28d96f9928 => generated 194 bytes in 52 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:08:01.804891 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:01.805434 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:01.812206 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:01.812692 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:01.813123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:01.813858 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:01.814310 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:01.814731 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:01.909231 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lazy-loading 'services' on Instance uuid a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:08:01.994326 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-7346b242-abc7-4fe7-b9fa-f27b648bd553 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1b5834f9-c7ed-4147-82f7-fe28d96f9928" status: 202 len: 0 microversion: 2.1 time: 0.193192 Sep 25 01:08:01.994938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 513/1015] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:08:01 2022] DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1b5834f9-c7ed-4147-82f7-fe28d96f9928 => generated 0 bytes in 195 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:08:04.211602 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:04.212363 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:04.219999 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:04.220558 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:04.221043 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:04.221860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:04.222403 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:04.222878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:04.258833 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-df990f3e-3b61-4657-bd5d-d032dcca7d27 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1607a9b6-f846-4a40-ba45-33a91b4898c1" status: 200 len: 194 microversion: 2.1 time: 0.051372 Sep 25 01:08:04.259469 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 503/1016] 10.222.0.52 () {58 vars in 1413 bytes} [Sun Sep 25 01:08:04 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1607a9b6-f846-4a40-ba45-33a91b4898c1 => generated 194 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:08:04.271617 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:04.272269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:04.281630 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:04.281855 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:04.282167 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:04.283493 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:04.283996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:04.284779 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:04.385598 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lazy-loading 'services' on Instance uuid a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:08:04.493048 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c2ea4e76-64c9-4836-81ad-3e35a9e6dfc4 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1607a9b6-f846-4a40-ba45-33a91b4898c1" status: 202 len: 0 microversion: 2.1 time: 0.226157 Sep 25 01:08:04.493776 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 514/1017] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:08:04 2022] DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments/1607a9b6-f846-4a40-ba45-33a91b4898c1 => generated 0 bytes in 228 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:08:05.717980 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:05.720184 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:05.727305 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:05.727805 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:05.728249 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:05.728976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:05.729490 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:05.729927 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:05.764186 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-38df6a0d-d9b4-4987-9616-a1493278567a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.051149 Sep 25 01:08:05.764799 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 504/1018] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:08:05 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments => generated 25 bytes in 53 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 25 01:08:05.896826 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:05.899189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:05.907695 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:05.908332 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:05.908886 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:05.909995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:05.910572 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:05.911111 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:05.950185 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-bbb8171e-984a-450e-ba24-81ec6676c25c tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments" status: 200 len: 25 microversion: 2.1 time: 0.057866 Sep 25 01:08:05.950803 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 515/1019] 10.222.0.52 () {58 vars in 1302 bytes} [Sun Sep 25 01:08:05 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4/os-volume_attachments => generated 25 bytes in 59 msecs (HTTP/1.1 200) 9 headers in 356 bytes (2 switches on core 0) Sep 25 01:08:06.055791 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:06.056653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:06.068608 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:06.069432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:06.070274 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:06.071588 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:06.072423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:06.073109 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:06.106419 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:08:06.200953 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-44dba92e-a2dd-4d1c-a568-d01ac5f69e5e tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 204 len: 0 microversion: 2.1 time: 0.151577 Sep 25 01:08:06.202136 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 505/1020] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:08:06 2022] DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:08:06.214039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:06.214836 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:06.222959 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:06.223603 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:06.224145 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:06.224954 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:06.225557 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:06.226083 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:06.375560 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:06.384120 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-aa5de87b-b7e3-4f57-80c0-af32fac566b8 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1674 microversion: 2.1 time: 0.176924 Sep 25 01:08:06.384711 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 516/1021] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:06 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1674 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:08:07.409691 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:07.410436 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:07.417544 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:07.418090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:07.419215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:07.420163 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:07.420681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:07.421155 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:07.586894 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:07.597141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c65e8d7e-5b3b-42fb-8264-560a6e2fc7ca tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1674 microversion: 2.1 time: 0.191867 Sep 25 01:08:07.597988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 506/1022] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:07 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1674 bytes in 194 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:08.628842 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:08.631213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:08.644490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:08.645439 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:08.646450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:08.647794 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:08.648619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:08.649410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:08.846410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:08.861733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-070b1026-249d-4c8a-8a0d-84658eacf34a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1674 microversion: 2.1 time: 0.240731 Sep 25 01:08:08.863334 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 517/1023] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:08 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1674 bytes in 244 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:09.364870 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-057d72a3-ac0a-42c6-989f-11230a051166 req-579d38ba-721d-4773-a3a9-b9db1a69798f service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "a098f4d1-9c50-4e95-a5aa-e7cb07850ad4", "name": "network-vif-deleted", "tag": "1efe19b6-e801-45ed-8aac-c94c3a3e0758"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:08:09.382247 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-057d72a3-ac0a-42c6-989f-11230a051166 req-579d38ba-721d-4773-a3a9-b9db1a69798f service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:09.383284 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-057d72a3-ac0a-42c6-989f-11230a051166 req-579d38ba-721d-4773-a3a9-b9db1a69798f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:09.384319 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-057d72a3-ac0a-42c6-989f-11230a051166 req-579d38ba-721d-4773-a3a9-b9db1a69798f service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:09.410330 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-057d72a3-ac0a-42c6-989f-11230a051166 req-579d38ba-721d-4773-a3a9-b9db1a69798f service nova] Creating event network-vif-deleted:1efe19b6-e801-45ed-8aac-c94c3a3e0758 for instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 on hv-cinder-maste Sep 25 01:08:09.420854 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-057d72a3-ac0a-42c6-989f-11230a051166 req-579d38ba-721d-4773-a3a9-b9db1a69798f service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.059622 Sep 25 01:08:09.422081 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 507/1024] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:08:09 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 62 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:08:09.891378 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:09.892403 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:09.900823 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:09.901553 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:09.902103 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:09.903187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:09.904076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:09.904605 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:09.945985 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:09.946611 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:09.949015 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:09.949015 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lazy-loading 'fault' on Instance uuid a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:08:10.026226 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '1e515c9c8dcf4a1aa7dd74fb7655adf4', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'f37b53a21b19482b8dcac6c38505a50b', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:10.036049 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-460a97c7-f2f8-4e71-bbb6-4d178986ffd3 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 200 len: 1328 microversion: 2.1 time: 0.150324 Sep 25 01:08:10.036681 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 518/1025] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:09 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 1328 bytes in 152 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:08:11.063844 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:11.064598 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:11.071531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:11.072036 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:11.072466 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:11.073193 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:11.073696 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:11.074133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:11.090655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] HTTP exception thrown: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Request method failure captured: Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 HTTP/1.1 Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6jzQ8ZGzZbsRNfl9mXkdLfXWHvwRSpA0SXP7A_oXBAYMbdKJOjlYvr8B_BfwcdUtJJN8BI8SPqL-aMgg5q4U3v5SVXcxzP40NPeDSdsurJSB5u579RBlZFTKMs4VmEMmE5E9fooDCVVv5dB6TpRCVhQnau9VkVTaJagbdgobkhvLXc Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:08:11.093000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f37b53a21b19482b8dcac6c38505a50b", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}] Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 1e515c9c8dcf4a1aa7dd74fb7655adf4 Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:11.096248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'a098f4d1-9c50-4e95-a5aa-e7cb07850ad4'} Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:08:11.098027 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:11.100866 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:11.103200 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Returning 404 to user: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:08:11.103200 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-322d3ef7-6d5c-41c3-9e56-17bafb38a4cf tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 404 len: 111 microversion: 2.1 time: 0.037249 Sep 25 01:08:11.103200 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 508/1026] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:11 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 111 bytes in 39 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:08:11.109434 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-55d9bcf3-844e-4012-a309-e22ad1b7031f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '.version_select at 0x7fe22e62f3a0>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:11.132706 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-55d9bcf3-844e-4012-a309-e22ad1b7031f tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/os-keypairs/tempest-keypair-1546838375" status: 202 len: 0 microversion: 2.1 time: 0.028584 Sep 25 01:08:11.133337 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 519/1027] 10.222.0.52 () {58 vars in 1221 bytes} [Sun Sep 25 01:08:11 2022] DELETE /compute/v2.1/os-keypairs/tempest-keypair-1546838375 => generated 0 bytes in 30 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:08:14.256662 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:14.257599 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:14.260309 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=26,nova_api:UPDATE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:14.267130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:14.267761 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:14.268244 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:14.269000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:14.269595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:14.270093 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:14.274781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=64,nova_cell1:UPDATE=3,nova_cell1:INSERT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:14.297823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] HTTP exception thrown: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Request method failure captured: Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 HTTP/1.1 Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6jzQ8ZGzZbsRNfl9mXkdLfXWHvwRSpA0SXP7A_oXBAYMbdKJOjlYvr8B_BfwcdUtJJN8BI8SPqL-aMgg5q4U3v5SVXcxzP40NPeDSdsurJSB5u579RBlZFTKMs4VmEMmE5E9fooDCVVv5dB6TpRCVhQnau9VkVTaJagbdgobkhvLXc Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:08:14.299746 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f37b53a21b19482b8dcac6c38505a50b", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}] Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 1e515c9c8dcf4a1aa7dd74fb7655adf4 Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.304125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': 'a098f4d1-9c50-4e95-a5aa-e7cb07850ad4'} Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:08:14.305657 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.308003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.312016 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Returning 404 to user: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:08:14.312016 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-39288878-b634-4998-b493-9acf308e9d0a tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 404 len: 111 microversion: 2.1 time: 0.049580 Sep 25 01:08:14.312016 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 509/1028] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:08:14 2022] DELETE /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 111 bytes in 52 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:08:14.316071 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:14.316784 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: a098f4d1-9c50-4e95-a5aa-e7cb07850ad4] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:14.319653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=24,nova_api:DELETE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:14.330578 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:14.331449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:14.332228 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:14.333547 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:14.334380 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:14.335128 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:14.344716 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=68,nova_cell1:UPDATE=2,nova_cell1:INSERT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:14.364876 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] HTTP exception thrown: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Request method failure captured: Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 HTTP/1.1 Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6jzQ8ZGzZbsRNfl9mXkdLfXWHvwRSpA0SXP7A_oXBAYMbdKJOjlYvr8B_BfwcdUtJJN8BI8SPqL-aMgg5q4U3v5SVXcxzP40NPeDSdsurJSB5u579RBlZFTKMs4VmEMmE5E9fooDCVVv5dB6TpRCVhQnau9VkVTaJagbdgobkhvLXc Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:08:14.368086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f37b53a21b19482b8dcac6c38505a50b", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}] Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 1e515c9c8dcf4a1aa7dd74fb7655adf4 Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.370313 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': 'a098f4d1-9c50-4e95-a5aa-e7cb07850ad4'} Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:08:14.373397 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. Sep 25 01:08:14.376450 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:14.378442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Returning 404 to user: Instance a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:08:14.378442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f79fabf3-7587-4c0b-839f-74ef70036512 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4" status: 404 len: 111 microversion: 2.1 time: 0.059299 Sep 25 01:08:14.378442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 520/1029] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:14 2022] GET /compute/v2.1/servers/a098f4d1-9c50-4e95-a5aa-e7cb07850ad4 => generated 111 bytes in 63 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:08:15.555556 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:15.556185 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:15.564003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:15.564494 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:15.564952 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:15.565723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:15.566181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:15.566607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:15.582302 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] HTTP exception thrown: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Request method failure captured: Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: DELETE /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 HTTP/1.1 Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6jzQ8ZGzZbsRNfl9mXkdLfXWHvwRSpA0SXP7A_oXBAYMbdKJOjlYvr8B_BfwcdUtJJN8BI8SPqL-aMgg5q4U3v5SVXcxzP40NPeDSdsurJSB5u579RBlZFTKMs4VmEMmE5E9fooDCVVv5dB6TpRCVhQnau9VkVTaJagbdgobkhvLXc Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:08:15.584076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f37b53a21b19482b8dcac6c38505a50b", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}] Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 1e515c9c8dcf4a1aa7dd74fb7655adf4 Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.586532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '1de127ae-8886-4d15-a6be-7b68902ccde8'} Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:08:15.587916 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 1095, in delete Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi self._delete(req.environ['nova.context'], req, id) Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 890, in _delete Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server(context, req, instance_uuid) Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.591021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.593678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Returning 404 to user: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:08:15.593678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-008f4ea3-5b25-4b66-a78f-776a3f0f16fb tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "DELETE /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 404 len: 111 microversion: 2.1 time: 0.034712 Sep 25 01:08:15.593678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 510/1030] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:08:15 2022] DELETE /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:08:15.597541 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:15.598152 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] [instance: 1de127ae-8886-4d15-a6be-7b68902ccde8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:15.607173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:15.607660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:15.608098 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:15.608817 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:15.609289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:15.609725 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:15.625944 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] HTTP exception thrown: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Request method failure captured: Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 HTTP/1.1 Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6jzQ8ZGzZbsRNfl9mXkdLfXWHvwRSpA0SXP7A_oXBAYMbdKJOjlYvr8B_BfwcdUtJJN8BI8SPqL-aMgg5q4U3v5SVXcxzP40NPeDSdsurJSB5u579RBlZFTKMs4VmEMmE5E9fooDCVVv5dB6TpRCVhQnau9VkVTaJagbdgobkhvLXc Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:08:15.627501 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_f37b53a21b19482b8dcac6c38505a50b", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/f37b53a21b19482b8dcac6c38505a50b"}]}] Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: f37b53a21b19482b8dcac6c38505a50b Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-AttachVolumeTestJSON-105378569 Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 1e515c9c8dcf4a1aa7dd74fb7655adf4 Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-AttachVolumeTestJSON-105378569-project Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.630118 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '1de127ae-8886-4d15-a6be-7b68902ccde8'} Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:08:15.631859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. Sep 25 01:08:15.634593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:15.636962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] Returning 404 to user: Instance 1de127ae-8886-4d15-a6be-7b68902ccde8 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:08:15.636962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-46d76b6d-22ca-4973-9b11-6764e85f7ff9 tempest-AttachVolumeTestJSON-105378569 tempest-AttachVolumeTestJSON-105378569-project] 10.222.0.52 "GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8" status: 404 len: 111 microversion: 2.1 time: 0.035950 Sep 25 01:08:15.636962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 521/1031] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:15 2022] GET /compute/v2.1/servers/1de127ae-8886-4d15-a6be-7b68902ccde8 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:08:41.947463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-1557918015", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "networks": [{"uuid": "bf2b84f7-77ae-4b11-ada8-9925bb081495"}]}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:08:41.958650 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:41.959444 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=3 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:42.030956 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] validate_networks() for [('bf2b84f7-77ae-4b11-ada8-9925bb081495', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:08:42.513875 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:08:42.514306 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:08:42.514792 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:08:42.515197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:08:42.515613 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:08:42.526037 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:42.526689 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:42.527962 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:42.528943 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:42.529511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:42.530058 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:42.551639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Getting quotas for project da4aa8615ed349f684f1ea9074a4bf6e. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:08:42.557568 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Getting quotas for user 7b14962aa90e4c0caceb538b43ebd2d8 and project da4aa8615ed349f684f1ea9074a4bf6e. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:08:42.574727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:08:42.577462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:08:42.636609 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e67ebdf1-4827-4893-ba4d-9557c16ade70 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.693283 Sep 25 01:08:42.637575 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 511/1032] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:08:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 695 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 01:08:42.657955 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-90a1946c-bd49-4db3-9785-c679592a498c tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:42.659010 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-90a1946c-bd49-4db3-9785-c679592a498c tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:42.664116 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:42.664593 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:42.744234 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-90a1946c-bd49-4db3-9785-c679592a498c tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:42.757487 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-90a1946c-bd49-4db3-9785-c679592a498c tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1214 microversion: 2.1 time: 0.108320 Sep 25 01:08:42.758480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 522/1033] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:42 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1214 bytes in 111 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:43.785426 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:43.786220 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:43.794325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:43.794828 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:43.795273 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:43.795997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:43.796643 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:43.797084 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:43.870846 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:43.879507 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7b60b4d5-5f9e-4151-a55b-9833ea761ed5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1266 microversion: 2.1 time: 0.098667 Sep 25 01:08:43.880094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 512/1034] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:43 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1266 bytes in 100 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:44.905975 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:44.906862 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:44.915440 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:44.915949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:44.916394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:44.918146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:44.918146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:44.918146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:45.115587 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:45.125911 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-014d3195-ca14-4ee7-8146-fb946c6bcd93 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1314 microversion: 2.1 time: 0.225368 Sep 25 01:08:45.126753 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 523/1035] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:44 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1314 bytes in 228 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:45.802277 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-b2f52aa9-af7d-48ce-aa1f-a91f415c2c81 req-fbca2410-0f32-4412-8a16-005f0e666548 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "47140790-3a76-434f-b15f-20c16caa1ac0", "tag": "b749a5de-4349-46be-814d-267b61200e7f"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:08:45.830607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b2f52aa9-af7d-48ce-aa1f-a91f415c2c81 req-fbca2410-0f32-4412-8a16-005f0e666548 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:45.831595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b2f52aa9-af7d-48ce-aa1f-a91f415c2c81 req-fbca2410-0f32-4412-8a16-005f0e666548 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:45.832365 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-b2f52aa9-af7d-48ce-aa1f-a91f415c2c81 req-fbca2410-0f32-4412-8a16-005f0e666548 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:45.855933 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-b2f52aa9-af7d-48ce-aa1f-a91f415c2c81 req-fbca2410-0f32-4412-8a16-005f0e666548 service nova] Creating event network-changed:b749a5de-4349-46be-814d-267b61200e7f for instance 47140790-3a76-434f-b15f-20c16caa1ac0 on hv-cinder-maste Sep 25 01:08:45.865918 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-b2f52aa9-af7d-48ce-aa1f-a91f415c2c81 req-fbca2410-0f32-4412-8a16-005f0e666548 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.069749 Sep 25 01:08:45.867277 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 513/1036] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:08:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 72 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:08:46.155496 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:46.156380 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:46.163660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:46.164207 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:46.164814 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:46.165628 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:46.166099 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:46.166542 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:46.384634 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:46.396699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-822e6bae-4169-44c6-9be4-dd3446771742 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1314 microversion: 2.1 time: 0.245566 Sep 25 01:08:46.397693 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 524/1037] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:46 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1314 bytes in 247 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:08:47.425731 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:47.426514 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:47.433957 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:47.434560 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:47.435071 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:47.435914 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:47.436390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:47.436825 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:47.727890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:47.738964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-afc0fb15-f7ea-432e-91cf-89a3a7d7ef1b tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1314 microversion: 2.1 time: 0.318476 Sep 25 01:08:47.739569 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 514/1038] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:47 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1314 bytes in 320 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:48.763269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:48.763889 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:48.771512 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:48.772005 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:48.772443 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:48.773164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:48.773661 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:48.774088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:48.954896 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:48.966691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ae8e8e84-5005-4fd7-a41c-4172fdf4acea tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1314 microversion: 2.1 time: 0.207950 Sep 25 01:08:48.967643 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 525/1039] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:48 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1314 bytes in 210 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:08:50.007456 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:50.008622 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:50.021859 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:50.022576 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:50.023309 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:50.024523 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:50.025390 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:50.026168 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:50.185393 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:50.195611 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-613dbc41-de06-4b49-afb5-8b8a79ac9beb tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1314 microversion: 2.1 time: 0.195547 Sep 25 01:08:50.196212 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 515/1040] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:49 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1314 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:50.470415 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-2e2bfe64-4ac4-4052-9651-accd381dd587 req-e0062e17-9c0f-46d2-8a22-e907b9ca6437 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "47140790-3a76-434f-b15f-20c16caa1ac0", "name": "network-vif-plugged", "status": "completed", "tag": "b749a5de-4349-46be-814d-267b61200e7f"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:08:50.490164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2e2bfe64-4ac4-4052-9651-accd381dd587 req-e0062e17-9c0f-46d2-8a22-e907b9ca6437 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:50.490337 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2e2bfe64-4ac4-4052-9651-accd381dd587 req-e0062e17-9c0f-46d2-8a22-e907b9ca6437 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:50.490763 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-2e2bfe64-4ac4-4052-9651-accd381dd587 req-e0062e17-9c0f-46d2-8a22-e907b9ca6437 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:50.509434 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-2e2bfe64-4ac4-4052-9651-accd381dd587 req-e0062e17-9c0f-46d2-8a22-e907b9ca6437 service nova] Creating event network-vif-plugged:b749a5de-4349-46be-814d-267b61200e7f for instance 47140790-3a76-434f-b15f-20c16caa1ac0 on hv-cinder-maste Sep 25 01:08:50.515474 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-2e2bfe64-4ac4-4052-9651-accd381dd587 req-e0062e17-9c0f-46d2-8a22-e907b9ca6437 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.052976 Sep 25 01:08:50.516158 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 526/1041] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:08:50 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 55 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:08:51.222785 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:51.223470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:51.230605 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:51.231100 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:51.231546 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:51.232273 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:51.232741 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:51.233199 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:51.403990 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:51.413654 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-8f66022f-fdcf-43bd-97be-58151a574b5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1314 microversion: 2.1 time: 0.195153 Sep 25 01:08:51.414324 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 516/1042] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:51 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1314 bytes in 197 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:52.441815 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:52.442721 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:52.453022 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:52.453619 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:52.454107 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:52.455610 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:52.456176 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:52.456655 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:52.653505 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:52.666581 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9564cdf0-4c58-4d20-a892-8f553355a998 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1498 microversion: 2.1 time: 0.230641 Sep 25 01:08:52.669164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 527/1043] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:52 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1498 bytes in 236 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:08:53.235729 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:53.236391 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:53.239462 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:53.240454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=6 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:53.248324 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:53.248878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:53.249540 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:53.250326 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:53.250844 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:53.251322 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:53.281468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:08:53.413754 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-101d6946-949e-4b67-9d8a-3d1682ea9bff tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "DELETE /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 204 len: 0 microversion: 2.1 time: 0.181689 Sep 25 01:08:53.414243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 517/1044] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:08:53 2022] DELETE /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 25 01:08:53.422804 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:53.423442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:53.425870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:08:53.431991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:53.432492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:53.433502 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:53.434333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:53.434697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:53.435135 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:53.595101 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:53.603884 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-66a1218d-daa7-46c0-b809-406bed8ff266 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1504 microversion: 2.1 time: 0.184925 Sep 25 01:08:53.604477 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 528/1045] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:53 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1504 bytes in 186 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:54.632988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:54.634237 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:54.645152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:54.645686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:54.646131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:54.646877 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:54.647332 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:54.647759 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:54.843091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:54.852090 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-9ac1e21c-e533-44d6-8f91-c5228982a4e2 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1504 microversion: 2.1 time: 0.226691 Sep 25 01:08:54.852677 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 518/1046] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:54 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1504 bytes in 229 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:55.668092 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-313d3e17-6325-42b2-9124-e8ac6fdd48a0 req-a9f93294-7e87-4a3f-ad1b-33554dff6565 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "47140790-3a76-434f-b15f-20c16caa1ac0", "name": "network-vif-deleted", "tag": "b749a5de-4349-46be-814d-267b61200e7f"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:08:55.684114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-313d3e17-6325-42b2-9124-e8ac6fdd48a0 req-a9f93294-7e87-4a3f-ad1b-33554dff6565 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:55.684592 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-313d3e17-6325-42b2-9124-e8ac6fdd48a0 req-a9f93294-7e87-4a3f-ad1b-33554dff6565 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:55.685925 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-313d3e17-6325-42b2-9124-e8ac6fdd48a0 req-a9f93294-7e87-4a3f-ad1b-33554dff6565 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:55.700660 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-313d3e17-6325-42b2-9124-e8ac6fdd48a0 req-a9f93294-7e87-4a3f-ad1b-33554dff6565 service nova] Creating event network-vif-deleted:b749a5de-4349-46be-814d-267b61200e7f for instance 47140790-3a76-434f-b15f-20c16caa1ac0 on hv-cinder-maste Sep 25 01:08:55.708848 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-313d3e17-6325-42b2-9124-e8ac6fdd48a0 req-a9f93294-7e87-4a3f-ad1b-33554dff6565 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.046069 Sep 25 01:08:55.709767 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 529/1047] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:08:55 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 48 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:08:55.878213 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:55.879281 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:55.890210 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:55.891021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:55.891751 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:55.892963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:55.893857 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:55.894596 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:55.938392 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:55.938887 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:55.939359 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:55.939822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lazy-loading 'fault' on Instance uuid 47140790-3a76-434f-b15f-20c16caa1ac0 {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:08:55.986658 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:55.995362 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-78447cc2-d0c0-4ce6-b9f8-5eb0cb83f9b5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 200 len: 1304 microversion: 2.1 time: 0.122578 Sep 25 01:08:55.996149 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 519/1048] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:55 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 1304 bytes in 124 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:57.021521 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:57.022292 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 47140790-3a76-434f-b15f-20c16caa1ac0] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:57.029073 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:57.029594 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:57.030033 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:57.030757 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:57.031215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:57.031637 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:57.046972 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] HTTP exception thrown: Instance 47140790-3a76-434f-b15f-20c16caa1ac0 could not be found. Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Request method failure captured: Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 HTTP/1.1 Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6oY5WhFChFqetuVY7zxw7F7DZCd0B2x__0Qb__c9TKFkRe9QRUIpq5b-EnF2G4eMI9wVXM4QOrpFctl2E03LCdyRSl_uKvKWpPw8ZPcznD40vl4ttK2uXWxgrvvsgHiDFpHnNAyeeL90qj9ADANfz7zhSow0CR4XJDHOA7mLo-O56Y Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: da4aa8615ed349f684f1ea9074a4bf6e Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-VolumesActionsTest-1480353271 Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:08:57.048667 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_da4aa8615ed349f684f1ea9074a4bf6e", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/da4aa8615ed349f684f1ea9074a4bf6e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/da4aa8615ed349f684f1ea9074a4bf6e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/da4aa8615ed349f684f1ea9074a4bf6e"}]}] Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-VolumesActionsTest-1480353271 Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: da4aa8615ed349f684f1ea9074a4bf6e Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-VolumesActionsTest-1480353271 Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-VolumesActionsTest-1480353271-project Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: 7b14962aa90e4c0caceb538b43ebd2d8 Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-VolumesActionsTest-1480353271-project Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 47140790-3a76-434f-b15f-20c16caa1ac0 could not be found. Sep 25 01:08:57.051215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '47140790-3a76-434f-b15f-20c16caa1ac0'} Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 47140790-3a76-434f-b15f-20c16caa1ac0 could not be found. Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:08:57.053004 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 47140790-3a76-434f-b15f-20c16caa1ac0 could not be found. Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 47140790-3a76-434f-b15f-20c16caa1ac0 could not be found. Sep 25 01:08:57.055938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:08:57.059307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Returning 404 to user: Instance 47140790-3a76-434f-b15f-20c16caa1ac0 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:08:57.059307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9cd36106-af97-4307-8e4a-f9c862debd75 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0" status: 404 len: 111 microversion: 2.1 time: 0.033571 Sep 25 01:08:57.059307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 530/1049] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:57 2022] GET /compute/v2.1/servers/47140790-3a76-434f-b15f-20c16caa1ac0 => generated 111 bytes in 35 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:08:57.124292 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesActionsTest-instance-66145811", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "networks": [{"uuid": "bf2b84f7-77ae-4b11-ada8-9925bb081495"}]}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:08:57.187824 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.network.neutron [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] validate_networks() for [('bf2b84f7-77ae-4b11-ada8-9925bb081495', None, None, None, None, None)] {{(pid=120450) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:08:57.602328 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Flavor limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:08:57.602781 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Image limits 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:08:57.603259 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Flavor pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:08:57.603684 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Image pref 0:0:0 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:08:57.604125 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.virt.hardware [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120450) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:08:57.615379 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:57.615868 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:57.616308 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:57.617065 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:57.617567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:57.617999 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:57.639415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Getting quotas for project da4aa8615ed349f684f1ea9074a4bf6e. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:08:57.644527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.quota [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Getting quotas for user 7b14962aa90e4c0caceb538b43ebd2d8 and project da4aa8615ed349f684f1ea9074a4bf6e. Resources: {'ram', 'cores', 'instances'} {{(pid=120450) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:08:57.658220 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Going to run 1 instances... {{(pid=120450) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:08:57.660616 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120450) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:08:57.714479 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-fcda96b2-0355-45ed-9671-3b4217cb7429 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.594680 Sep 25 01:08:57.715344 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 520/1050] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:08:57 2022] POST /compute/v2.1/servers => generated 380 bytes in 597 msecs (HTTP/1.1 202) 10 headers in 451 bytes (2 switches on core 0) Sep 25 01:08:57.727733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d0864ee1-779b-40a8-8c34-64b8a93e9eb4 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:57.728351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d0864ee1-779b-40a8-8c34-64b8a93e9eb4 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:57.808156 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d0864ee1-779b-40a8-8c34-64b8a93e9eb4 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:57.816993 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d0864ee1-779b-40a8-8c34-64b8a93e9eb4 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1212 microversion: 2.1 time: 0.093116 Sep 25 01:08:57.817880 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 531/1051] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:57 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1212 bytes in 95 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:08:58.856436 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:08:58.857603 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:08:58.871042 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:58.871949 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:58.872751 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:58.874066 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:08:58.874937 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:08:58.875724 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:08:58.955129 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:08:58.964860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-206687cb-297d-49a2-b4e3-863af2f9d418 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1208 microversion: 2.1 time: 0.117065 Sep 25 01:08:58.966261 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 521/1052] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:58 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1208 bytes in 120 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:00.003786 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:00.004750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:00.018969 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:00.019811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:00.020626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:00.022047 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:00.022947 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:00.023792 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:00.140650 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:00.156674 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-17b5a535-ff48-49ea-9a81-a642954031c5 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1270 microversion: 2.1 time: 0.157519 Sep 25 01:09:00.157835 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 532/1053] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:08:59 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1270 bytes in 160 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:01.183706 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:01.185081 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:01.192277 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:01.192925 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:01.193563 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:01.194379 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:01.194869 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:01.195340 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:01.440752 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:01.450425 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4348d1aa-150f-4719-b4bf-1f99dceaeae0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1312 microversion: 2.1 time: 0.271573 Sep 25 01:09:01.465289 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 522/1054] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:01 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1312 bytes in 274 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:01.741806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-426f69a7-43b0-435f-bc57-a3b2504ba91e req-473ddc33-9b7f-4eac-aec5-7a046e8adfad service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1cb9e557-600c-49fa-bcd2-d191a7c494bc", "tag": "0f50e565-aa39-40fb-b6f7-e3feaa6f231f"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:09:01.762255 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-426f69a7-43b0-435f-bc57-a3b2504ba91e req-473ddc33-9b7f-4eac-aec5-7a046e8adfad service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:01.763404 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-426f69a7-43b0-435f-bc57-a3b2504ba91e req-473ddc33-9b7f-4eac-aec5-7a046e8adfad service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:01.764112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-426f69a7-43b0-435f-bc57-a3b2504ba91e req-473ddc33-9b7f-4eac-aec5-7a046e8adfad service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:01.781527 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-426f69a7-43b0-435f-bc57-a3b2504ba91e req-473ddc33-9b7f-4eac-aec5-7a046e8adfad service nova] Creating event network-changed:0f50e565-aa39-40fb-b6f7-e3feaa6f231f for instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc on hv-cinder-maste Sep 25 01:09:01.788094 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-426f69a7-43b0-435f-bc57-a3b2504ba91e req-473ddc33-9b7f-4eac-aec5-7a046e8adfad service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.051881 Sep 25 01:09:01.789066 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 533/1055] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:09:01 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:09:02.475730 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:02.476384 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:02.483860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:02.484529 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:02.485029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:02.485966 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:02.486575 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:02.487193 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:02.646094 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:02.655257 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ff5d3acd-0260-4c4f-a783-65b38d7f4253 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1312 microversion: 2.1 time: 0.183440 Sep 25 01:09:02.655919 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 523/1056] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:02 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1312 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:03.681380 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:03.682693 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:03.693178 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:03.693949 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:03.694584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:03.695614 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:03.696243 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:03.697021 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:03.886682 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:03.895424 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-998990fa-ded8-4e8e-8e25-4019b410df13 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1312 microversion: 2.1 time: 0.220803 Sep 25 01:09:03.896022 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 534/1057] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:03 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1312 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:04.921422 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:04.922483 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:04.931075 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:04.931753 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:04.932342 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:04.933338 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:04.934148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:04.934758 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:05.107668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:05.116861 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-376f588b-9f00-4caa-8bff-eb0ccb6184cd tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1312 microversion: 2.1 time: 0.200638 Sep 25 01:09:05.117541 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 524/1058] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:04 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1312 bytes in 202 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:06.140633 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:06.141418 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:06.148528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:06.149019 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:06.149500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:06.150225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:06.150680 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:06.151105 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:06.313765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:06.322164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-99e0be15-b8b1-454d-a493-a0455bf6f962 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1312 microversion: 2.1 time: 0.185875 Sep 25 01:09:06.322767 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 535/1059] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:06 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1312 bytes in 187 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:06.525041 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-96d597d8-372b-4731-bdd4-4580633796bb req-fa93874d-c429-4400-a1bb-bd454e382b27 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1cb9e557-600c-49fa-bcd2-d191a7c494bc", "name": "network-vif-plugged", "status": "completed", "tag": "0f50e565-aa39-40fb-b6f7-e3feaa6f231f"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:09:06.539397 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-96d597d8-372b-4731-bdd4-4580633796bb req-fa93874d-c429-4400-a1bb-bd454e382b27 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:06.539890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-96d597d8-372b-4731-bdd4-4580633796bb req-fa93874d-c429-4400-a1bb-bd454e382b27 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:06.540337 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-96d597d8-372b-4731-bdd4-4580633796bb req-fa93874d-c429-4400-a1bb-bd454e382b27 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:06.556450 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-96d597d8-372b-4731-bdd4-4580633796bb req-fa93874d-c429-4400-a1bb-bd454e382b27 service nova] Creating event network-vif-plugged:0f50e565-aa39-40fb-b6f7-e3feaa6f231f for instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc on hv-cinder-maste Sep 25 01:09:06.563208 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-96d597d8-372b-4731-bdd4-4580633796bb req-fa93874d-c429-4400-a1bb-bd454e382b27 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.042575 Sep 25 01:09:06.563832 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 525/1060] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:09:06 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:09:07.346515 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:07.347146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:07.355459 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:07.355964 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:07.356402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:07.357124 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:07.357618 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:07.358049 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:07.515729 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:07.523937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e4acb2eb-900f-446f-91c6-a443f2d0666f tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1312 microversion: 2.1 time: 0.181937 Sep 25 01:09:07.524522 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 536/1061] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:07 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1312 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:08.547361 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:08.548434 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=6 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:08.554117 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:08.554959 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:08.564666 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:08.565229 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:08.565860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:08.566697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:08.567215 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:08.567680 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:08.738898 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:08.748353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c79a8620-70f4-4a77-ac87-1f39b5a95e5e tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1496 microversion: 2.1 time: 0.203251 Sep 25 01:09:08.748950 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 526/1062] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:08 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1496 bytes in 205 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:09.369062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:09.369788 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:09.372127 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:09.378703 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:09.379202 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:09.379643 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:09.380391 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:09.380848 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:09.381299 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:09.409670 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:09:09.514970 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-a658f9c1-636d-48f5-9e27-0e5df9f21744 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "DELETE /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 204 len: 0 microversion: 2.1 time: 0.149142 Sep 25 01:09:09.515688 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 537/1063] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:09:09 2022] DELETE /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:09:09.525438 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:09.526060 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:09.532897 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:09.533441 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:09.533888 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:09.534610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:09.535069 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:09.535500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:09.681802 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:09.690378 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-393cef7b-e858-4c64-b83c-b7f0ff1ac4f9 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1502 microversion: 2.1 time: 0.169271 Sep 25 01:09:09.690978 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 527/1064] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:09 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1502 bytes in 171 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:10.716510 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:10.717344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:10.724777 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:10.725368 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:10.725863 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:10.726621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:10.727394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:10.727958 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:10.884486 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:10.893029 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-11bbdc99-e43e-46ca-8fb3-3dfbf024aaf0 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1502 microversion: 2.1 time: 0.181710 Sep 25 01:09:10.893664 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 538/1065] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:10 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1502 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:11.862902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-060cb6de-6baa-4bd0-8c9d-747e83fcba58 req-8c1297a0-1868-459e-8016-331c6a38df95 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1cb9e557-600c-49fa-bcd2-d191a7c494bc", "name": "network-vif-deleted", "tag": "0f50e565-aa39-40fb-b6f7-e3feaa6f231f"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:09:11.879595 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-060cb6de-6baa-4bd0-8c9d-747e83fcba58 req-8c1297a0-1868-459e-8016-331c6a38df95 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:11.880441 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-060cb6de-6baa-4bd0-8c9d-747e83fcba58 req-8c1297a0-1868-459e-8016-331c6a38df95 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:11.881224 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-060cb6de-6baa-4bd0-8c9d-747e83fcba58 req-8c1297a0-1868-459e-8016-331c6a38df95 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:11.889704 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=98,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:11.906365 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-060cb6de-6baa-4bd0-8c9d-747e83fcba58 req-8c1297a0-1868-459e-8016-331c6a38df95 service nova] Creating event network-vif-deleted:0f50e565-aa39-40fb-b6f7-e3feaa6f231f for instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc on hv-cinder-maste Sep 25 01:09:11.913823 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-060cb6de-6baa-4bd0-8c9d-747e83fcba58 req-8c1297a0-1868-459e-8016-331c6a38df95 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.056827 Sep 25 01:09:11.915021 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 528/1066] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:09:11 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 60 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:09:11.919530 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:11.920284 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:11.929343 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:11.930000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:11.930541 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:11.931435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:11.932000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:11.932695 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:12.021234 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '7b14962aa90e4c0caceb538b43ebd2d8', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': 'da4aa8615ed349f684f1ea9074a4bf6e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:12.030874 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-015e89d2-3405-4278-bae0-f79aecfa6544 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 200 len: 1297 microversion: 2.1 time: 0.117470 Sep 25 01:09:12.031800 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 539/1067] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:11 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 1297 bytes in 119 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:13.058499 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:13.059218 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] [instance: 1cb9e557-600c-49fa-bcd2-d191a7c494bc] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:13.062197 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=66,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:13.071567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:13.072069 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:13.072513 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:13.073302 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:13.073756 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:13.074190 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:13.092666 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] HTTP exception thrown: Instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc could not be found. Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Request method failure captured: Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc HTTP/1.1 Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6oY5WhFChFqetuVY7zxw7F7DZCd0B2x__0Qb__c9TKFkRe9QRUIpq5b-EnF2G4eMI9wVXM4QOrpFctl2E03LCdyRSl_uKvKWpPw8ZPcznD40vl4ttK2uXWxgrvvsgHiDFpHnNAyeeL90qj9ADANfz7zhSow0CR4XJDHOA7mLo-O56Y Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: da4aa8615ed349f684f1ea9074a4bf6e Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesActionsTest-1480353271 Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:09:13.094467 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_da4aa8615ed349f684f1ea9074a4bf6e", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/da4aa8615ed349f684f1ea9074a4bf6e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/da4aa8615ed349f684f1ea9074a4bf6e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/da4aa8615ed349f684f1ea9074a4bf6e"}]}] Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesActionsTest-1480353271 Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: da4aa8615ed349f684f1ea9074a4bf6e Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesActionsTest-1480353271 Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesActionsTest-1480353271-project Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 7b14962aa90e4c0caceb538b43ebd2d8 Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesActionsTest-1480353271-project Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc could not be found. Sep 25 01:09:13.097360 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '1cb9e557-600c-49fa-bcd2-d191a7c494bc'} Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc could not be found. Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:09:13.098797 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc could not be found. Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc could not be found. Sep 25 01:09:13.101976 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:13.105138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] Returning 404 to user: Instance 1cb9e557-600c-49fa-bcd2-d191a7c494bc could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:09:13.105138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7896f5ea-d7d2-4b69-b9d8-7faaea559ae8 tempest-VolumesActionsTest-1480353271 tempest-VolumesActionsTest-1480353271-project] 10.222.0.52 "GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc" status: 404 len: 111 microversion: 2.1 time: 0.042084 Sep 25 01:09:13.105138 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 529/1068] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:13 2022] GET /compute/v2.1/servers/1cb9e557-600c-49fa-bcd2-d191a7c494bc => generated 111 bytes in 43 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:09:41.043076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesNegativeTest-instance-423929056", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:09:41.051217 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=45,nova_api:UPDATE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:41.052147 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=88,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:41.111096 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] validate_networks() for None {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:09:41.464676 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:09:41.465442 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:09:41.466146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:09:41.466829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:09:41.467333 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:09:41.482528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:41.483400 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:41.484185 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:41.485580 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:41.486412 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:41.487185 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:41.521853 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Getting quotas for project 46f8efea67784ad48d92d7ab27e04e5e. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:09:41.540620 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Getting quotas for user 9e0e13e4786c4de395d51409dcb48cb5 and project 46f8efea67784ad48d92d7ab27e04e5e. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:09:41.561881 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:09:41.566807 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:09:41.657401 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d1364d18-7fce-43d4-a94c-88a92b9bdc27 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.619042 Sep 25 01:09:41.658392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 540/1069] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:09:41 2022] POST /compute/v2.1/servers => generated 380 bytes in 621 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 01:09:41.675057 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-20d85250-de3d-4cf8-a6fd-ba11e4481c86 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:41.675917 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-20d85250-de3d-4cf8-a6fd-ba11e4481c86 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:41.678011 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 25 01:09:41.678255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Sun Sep 25 01:09:41 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e (ip 10.222.0.52) !!! Sep 25 01:09:41.680107 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:41.680401 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:41.754592 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-20d85250-de3d-4cf8-a6fd-ba11e4481c86 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:41.770486 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-20d85250-de3d-4cf8-a6fd-ba11e4481c86 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1214 microversion: 2.1 time: 0.103072 Sep 25 01:09:41.771602 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 530/1070] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:41 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1214 bytes in 106 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:42.811266 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:42.811991 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:42.818373 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:42.818870 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:42.819315 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:42.820033 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:42.820494 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:42.820925 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:42.893213 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:42.902524 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-88a719a8-fa0f-4cbf-a997-31740d6eccd8 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1266 microversion: 2.1 time: 0.096716 Sep 25 01:09:42.903164 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 541/1071] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:42 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1266 bytes in 98 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:43.927571 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:43.928678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:43.936973 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:43.937666 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:43.938224 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:43.939080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:43.939656 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:43.940201 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:44.019173 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:44.033531 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4a12bc11-895e-445a-8fb1-5fdaf37862f1 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1272 microversion: 2.1 time: 0.109685 Sep 25 01:09:44.034655 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 531/1072] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:43 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1272 bytes in 112 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:45.072823 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:45.073777 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:45.081008 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:45.081556 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:45.082078 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:45.082730 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:45.083232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:45.083666 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:45.199804 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-e72cbe2b-e35b-46b4-8754-7321058d73c9 req-ad164603-6e3e-4ba8-a709-660289cc07ee service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "1c0fe499-c590-48ca-b769-344dbabce44e", "tag": "58b26c08-ffe5-4779-8d5c-4db5e137c572"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:09:45.222221 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e72cbe2b-e35b-46b4-8754-7321058d73c9 req-ad164603-6e3e-4ba8-a709-660289cc07ee service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:45.223026 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e72cbe2b-e35b-46b4-8754-7321058d73c9 req-ad164603-6e3e-4ba8-a709-660289cc07ee service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:45.223778 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-e72cbe2b-e35b-46b4-8754-7321058d73c9 req-ad164603-6e3e-4ba8-a709-660289cc07ee service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:45.241237 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-e72cbe2b-e35b-46b4-8754-7321058d73c9 req-ad164603-6e3e-4ba8-a709-660289cc07ee service nova] Creating event network-changed:58b26c08-ffe5-4779-8d5c-4db5e137c572 for instance 1c0fe499-c590-48ca-b769-344dbabce44e on hv-cinder-maste Sep 25 01:09:45.251497 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-e72cbe2b-e35b-46b4-8754-7321058d73c9 req-ad164603-6e3e-4ba8-a709-660289cc07ee service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.059248 Sep 25 01:09:45.253619 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 532/1073] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:09:45 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 63 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:09:45.305161 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:45.315847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-ad890b9d-58a0-4e67-ba6e-3a283358979a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1314 microversion: 2.1 time: 0.247230 Sep 25 01:09:45.316792 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 542/1074] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:45 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1314 bytes in 249 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:46.348920 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:46.349666 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:46.357334 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:46.357838 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:46.358282 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:46.359014 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:46.359477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:46.359908 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:46.517137 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:46.525654 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e4126e80-f349-44fe-a701-c79e51b96d49 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1314 microversion: 2.1 time: 0.181685 Sep 25 01:09:46.526242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 533/1075] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:46 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1314 bytes in 183 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:47.549842 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:47.550709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:47.558646 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:47.559145 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:47.559582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:47.560289 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:47.560744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:47.561173 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:47.719806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:47.728670 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-92879204-3d0e-4112-a2c2-0df459c36cfb tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1314 microversion: 2.1 time: 0.183543 Sep 25 01:09:47.729293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 543/1076] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:47 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1314 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:48.755718 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:48.756493 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:48.765054 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:48.765545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:48.765981 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:48.766681 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:48.767222 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:48.767641 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:48.922039 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:48.930346 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-e4f66518-824f-4fb8-9316-7627e541a072 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1314 microversion: 2.1 time: 0.180000 Sep 25 01:09:48.930975 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 534/1077] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:48 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1314 bytes in 182 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:49.957347 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:49.958003 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:49.965406 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:49.965901 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:49.966338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:49.967268 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:49.967830 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:49.968304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:50.130957 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:50.140399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-51dd0d84-4308-4cc1-87bd-abe3a178912e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1314 microversion: 2.1 time: 0.188040 Sep 25 01:09:50.141000 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 544/1078] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:49 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1314 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:51.174904 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:51.175985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:51.185535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:51.186358 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:51.187070 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:51.188010 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:51.188588 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:51.189080 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:51.245585 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-8e27719e-cf33-43c8-af9c-ca74d17a4aa4 req-8923aaaf-08cb-47ab-80bf-08f2617c07de service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c0fe499-c590-48ca-b769-344dbabce44e", "name": "network-vif-plugged", "status": "completed", "tag": "58b26c08-ffe5-4779-8d5c-4db5e137c572"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:09:51.258203 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8e27719e-cf33-43c8-af9c-ca74d17a4aa4 req-8923aaaf-08cb-47ab-80bf-08f2617c07de service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:51.259082 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8e27719e-cf33-43c8-af9c-ca74d17a4aa4 req-8923aaaf-08cb-47ab-80bf-08f2617c07de service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:51.259713 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-8e27719e-cf33-43c8-af9c-ca74d17a4aa4 req-8923aaaf-08cb-47ab-80bf-08f2617c07de service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:51.277744 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-8e27719e-cf33-43c8-af9c-ca74d17a4aa4 req-8923aaaf-08cb-47ab-80bf-08f2617c07de service nova] Creating event network-vif-plugged:58b26c08-ffe5-4779-8d5c-4db5e137c572 for instance 1c0fe499-c590-48ca-b769-344dbabce44e on hv-cinder-maste Sep 25 01:09:51.283800 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-8e27719e-cf33-43c8-af9c-ca74d17a4aa4 req-8923aaaf-08cb-47ab-80bf-08f2617c07de service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.050945 Sep 25 01:09:51.285800 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 545/1079] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:09:51 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 54 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:09:51.381231 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:51.390429 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-d6a88d72-456d-4208-a805-d0b69399a03c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1314 microversion: 2.1 time: 0.220257 Sep 25 01:09:51.391822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 535/1080] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:51 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1314 bytes in 222 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:52.413458 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:52.414562 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=6 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:52.420390 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:52.421103 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:52.428784 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:52.429305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:52.429746 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:52.430461 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:52.430919 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:52.431343 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:52.605140 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:52.614154 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-5705449b-74e8-4dbb-aa89-a1ba08f8358e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1314 microversion: 2.1 time: 0.201435 Sep 25 01:09:52.614750 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 546/1081] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:52 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1314 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:53.638764 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:53.639545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:53.641585 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:09:53.647369 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:53.647860 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:53.648302 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:53.649000 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:53.649485 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:53.649910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:53.812890 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:53.820963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ffef7148-e942-47ff-b0a2-f5b324a8fba3 tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1463 microversion: 2.1 time: 0.186671 Sep 25 01:09:53.821881 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 536/1082] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:53 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1463 bytes in 188 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:09:53.871292 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:53.871872 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:53.879269 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:53.879811 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:53.880255 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:53.881047 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:53.881634 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:53.882062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:53.912488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:09:54.002035 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c2a2781a-bad4-4928-9fcc-9c3e81d29f8c tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "DELETE /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 204 len: 0 microversion: 2.1 time: 0.135225 Sep 25 01:09:54.002684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 547/1083] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:09:53 2022] DELETE /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:09:54.012154 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:54.012848 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:54.022293 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:54.023203 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:54.024077 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:54.025518 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:54.026365 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:54.027141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:54.178964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:54.187477 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c6a94245-3cf6-49e8-b60f-7b06bdbdf76e tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1469 microversion: 2.1 time: 0.179866 Sep 25 01:09:54.188088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 537/1084] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:54 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1469 bytes in 181 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:55.217770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:55.218482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:55.225433 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:55.225937 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:55.226410 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:55.227129 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:55.227584 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:55.228014 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:55.379707 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:55.389155 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-369c3c48-3033-48b0-a78d-a6ec6026b7ef tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1469 microversion: 2.1 time: 0.176191 Sep 25 01:09:55.389776 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 548/1085] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:55 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1469 bytes in 178 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:56.152276 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-ee673327-8381-4db4-bdf9-e8f4518eec89 req-efc89999-76a5-4bcc-babd-7734d5642f11 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "1c0fe499-c590-48ca-b769-344dbabce44e", "name": "network-vif-deleted", "tag": "58b26c08-ffe5-4779-8d5c-4db5e137c572"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:09:56.161977 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-ee673327-8381-4db4-bdf9-e8f4518eec89 req-efc89999-76a5-4bcc-babd-7734d5642f11 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:56.162476 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-ee673327-8381-4db4-bdf9-e8f4518eec89 req-efc89999-76a5-4bcc-babd-7734d5642f11 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:56.162902 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-ee673327-8381-4db4-bdf9-e8f4518eec89 req-efc89999-76a5-4bcc-babd-7734d5642f11 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:56.176209 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-ee673327-8381-4db4-bdf9-e8f4518eec89 req-efc89999-76a5-4bcc-babd-7734d5642f11 service nova] Creating event network-vif-deleted:58b26c08-ffe5-4779-8d5c-4db5e137c572 for instance 1c0fe499-c590-48ca-b769-344dbabce44e on hv-cinder-maste Sep 25 01:09:56.182525 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-ee673327-8381-4db4-bdf9-e8f4518eec89 req-efc89999-76a5-4bcc-babd-7734d5642f11 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.035030 Sep 25 01:09:56.183484 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 538/1086] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:09:56 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 37 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:09:56.415112 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:56.416764 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:56.425392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:56.425942 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:56.426625 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:56.427678 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:56.428255 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:56.428893 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:56.467435 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:56.467972 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:56.468411 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:56.468863 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lazy-loading 'fault' on Instance uuid 1c0fe499-c590-48ca-b769-344dbabce44e {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:09:56.515902 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': '9e0e13e4786c4de395d51409dcb48cb5', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '46f8efea67784ad48d92d7ab27e04e5e', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:09:56.526375 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-44093ead-305f-4fb8-b221-f73f33faaafd tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 200 len: 1304 microversion: 2.1 time: 0.116233 Sep 25 01:09:56.526977 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 549/1087] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:56 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 1304 bytes in 118 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:09:57.554187 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:09:57.555012 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] [instance: 1c0fe499-c590-48ca-b769-344dbabce44e] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:09:57.565222 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:57.565745 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:57.566452 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:57.567644 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:09:57.568175 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:09:57.568826 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:09:57.588468 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] HTTP exception thrown: Instance 1c0fe499-c590-48ca-b769-344dbabce44e could not be found. Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Request method failure captured: Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e HTTP/1.1 Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6pT__5Su1DZc2fNpdH0AhlwO9A2n62f0VPpqEPhaRq9DkuXm7wh-7-NLxmiOhAP8XcdGMNrcYaAuZXo2eyFXB8lo5k-HkH8Ee3i-yUHH5ZM_N1hj_Gvhb_cWmr7BbuAAt9D-nSFzG78KRJbeWpA6KdNWslLKH2eSx1ApXbzrkidqAI Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 46f8efea67784ad48d92d7ab27e04e5e Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesNegativeTest-2086733823 Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:09:57.590829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_46f8efea67784ad48d92d7ab27e04e5e", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/46f8efea67784ad48d92d7ab27e04e5e"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f8efea67784ad48d92d7ab27e04e5e"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/46f8efea67784ad48d92d7ab27e04e5e"}]}] Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesNegativeTest-2086733823 Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 46f8efea67784ad48d92d7ab27e04e5e Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesNegativeTest-2086733823 Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesNegativeTest-2086733823-project Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: 9e0e13e4786c4de395d51409dcb48cb5 Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesNegativeTest-2086733823-project Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 1c0fe499-c590-48ca-b769-344dbabce44e could not be found. Sep 25 01:09:57.592988 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '1c0fe499-c590-48ca-b769-344dbabce44e'} Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 1c0fe499-c590-48ca-b769-344dbabce44e could not be found. Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:09:57.594530 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 1c0fe499-c590-48ca-b769-344dbabce44e could not be found. Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 1c0fe499-c590-48ca-b769-344dbabce44e could not be found. Sep 25 01:09:57.597092 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:09:57.599668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] Returning 404 to user: Instance 1c0fe499-c590-48ca-b769-344dbabce44e could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:09:57.599668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-3d2de33b-06a3-4d5a-9e1c-90d1b4bdb53a tempest-VolumesNegativeTest-2086733823 tempest-VolumesNegativeTest-2086733823-project] 10.222.0.52 "GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e" status: 404 len: 111 microversion: 2.1 time: 0.043857 Sep 25 01:09:57.599668 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 539/1088] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:09:57 2022] GET /compute/v2.1/servers/1c0fe499-c590-48ca-b769-344dbabce44e => generated 111 bytes in 45 msecs (HTTP/1.1 404) 9 headers in 379 bytes (2 switches on core 0) Sep 25 01:10:05.728109 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-114831713", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "networks": [{"uuid": "567e0576-161f-495c-b882-012356a1e499"}]}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:10:05.790185 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] validate_networks() for [('567e0576-161f-495c-b882-012356a1e499', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:10:06.320490 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:10:06.320962 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:10:06.321564 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:10:06.321984 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:10:06.322457 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:10:06.332636 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:06.333133 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:06.333753 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:06.334520 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:06.334982 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:06.335413 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:06.357208 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Getting quotas for project 1f0ea61e4de94ce083c572843a3f09fa. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:10:06.362133 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Getting quotas for user d978fab9ac0a4094a7e1008337ed2af9 and project 1f0ea61e4de94ce083c572843a3f09fa. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:10:06.375068 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:10:06.377451 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:10:06.428423 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8dcc3a02-d9f0-40f6-924f-a26c67dc890f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.704773 Sep 25 01:10:06.429076 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 550/1089] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:10:05 2022] POST /compute/v2.1/servers => generated 380 bytes in 706 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 01:10:06.440818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-cb58ce6d-17cf-4bac-a88d-d3703e0d8a17 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:06.441678 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-cb58ce6d-17cf-4bac-a88d-d3703e0d8a17 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:06.528985 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-cb58ce6d-17cf-4bac-a88d-d3703e0d8a17 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:06.540588 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-cb58ce6d-17cf-4bac-a88d-d3703e0d8a17 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1218 microversion: 2.1 time: 0.103290 Sep 25 01:10:06.541610 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 540/1090] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:06 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1218 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:10:07.578350 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:07.579658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:07.587293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:07.587968 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:07.588420 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:07.589154 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:07.589650 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:07.590088 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:07.682516 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:07.697621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f208250a-32f8-47bf-bb20-637c12a37e41 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1270 microversion: 2.1 time: 0.124381 Sep 25 01:10:07.697621 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 551/1091] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:07 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1270 bytes in 126 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:10:08.725739 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:08.726910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:08.730364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=42 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:08.738539 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:08.739575 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:08.740415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:08.741744 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:08.742641 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:08.743288 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:09.016192 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:09.026148 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-dee7ad4d-f858-4e24-80c1-d6fd2bc411e1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1318 microversion: 2.1 time: 0.305521 Sep 25 01:10:09.026819 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 541/1092] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:08 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1318 bytes in 307 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:10:09.695675 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-214405a6-990f-480e-b62b-80420e17a234 req-b9307f8e-47da-4760-a266-9dad698e84a3 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "02806a84-8467-4b1c-9e5d-9044627225c8", "tag": "b6d5a291-fe8f-4e93-8b5f-7dfa9448f2b4"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:10:09.710462 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-214405a6-990f-480e-b62b-80420e17a234 req-b9307f8e-47da-4760-a266-9dad698e84a3 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:09.710956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-214405a6-990f-480e-b62b-80420e17a234 req-b9307f8e-47da-4760-a266-9dad698e84a3 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:09.711398 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-214405a6-990f-480e-b62b-80420e17a234 req-b9307f8e-47da-4760-a266-9dad698e84a3 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:09.727039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-214405a6-990f-480e-b62b-80420e17a234 req-b9307f8e-47da-4760-a266-9dad698e84a3 service nova] Creating event network-changed:b6d5a291-fe8f-4e93-8b5f-7dfa9448f2b4 for instance 02806a84-8467-4b1c-9e5d-9044627225c8 on hv-cinder-maste Sep 25 01:10:09.733686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-214405a6-990f-480e-b62b-80420e17a234 req-b9307f8e-47da-4760-a266-9dad698e84a3 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.042283 Sep 25 01:10:09.734394 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 552/1093] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:10:09 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 44 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:10:10.051722 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:10.052829 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:10.061447 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:10.062082 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:10.062639 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:10.063504 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:10.064182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:10.064725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:10.308029 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:10.317017 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-4a3bdcd5-e3f9-4a05-8ba4-07fde13d6253 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1318 microversion: 2.1 time: 0.269568 Sep 25 01:10:10.317970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 542/1094] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:10 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1318 bytes in 272 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:10:11.348747 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:11.350978 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:11.356765 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=56,nova_api:INSERT=6,nova_api:UPDATE=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:11.367399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:11.368264 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:11.370066 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:11.371484 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:11.372314 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:11.373073 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:11.382363 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=73,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:11.566622 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:11.583301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-8d8d1832-5e46-4952-b0e7-c18bf3399aa5 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1318 microversion: 2.1 time: 0.241760 Sep 25 01:10:11.584351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 553/1095] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:11 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1318 bytes in 245 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:10:12.608906 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:12.609751 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:12.612271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=31 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:12.619858 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:12.620426 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:12.620910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:12.621805 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:12.622432 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:12.622993 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:12.796511 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:12.805418 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c953e4db-93b5-4fb1-a867-b09d3b4ce811 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1318 microversion: 2.1 time: 0.201017 Sep 25 01:10:12.806624 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 543/1096] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:12 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1318 bytes in 203 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:10:13.829931 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:13.830829 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:13.838709 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:13.839296 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:13.839867 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:13.840701 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:13.841449 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:13.842114 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:14.289734 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:14.302656 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-41caf5d9-4f64-4d13-860c-8626b4326bd9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1318 microversion: 2.1 time: 0.476860 Sep 25 01:10:14.304267 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 554/1097] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:13 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1318 bytes in 479 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:10:15.342591 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:15.343719 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:15.355304 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:15.355818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:15.356333 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:15.357063 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:15.357569 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:15.358001 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:15.440686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-f87a0c66-bd64-41d7-9b05-392f4a708f81 req-cceb582c-d71b-4382-8c5c-7ffeec06b0f4 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02806a84-8467-4b1c-9e5d-9044627225c8", "name": "network-vif-plugged", "status": "completed", "tag": "b6d5a291-fe8f-4e93-8b5f-7dfa9448f2b4"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:10:15.455489 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-f87a0c66-bd64-41d7-9b05-392f4a708f81 req-cceb582c-d71b-4382-8c5c-7ffeec06b0f4 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:15.455976 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-f87a0c66-bd64-41d7-9b05-392f4a708f81 req-cceb582c-d71b-4382-8c5c-7ffeec06b0f4 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:15.456737 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-f87a0c66-bd64-41d7-9b05-392f4a708f81 req-cceb582c-d71b-4382-8c5c-7ffeec06b0f4 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:15.470833 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-f87a0c66-bd64-41d7-9b05-392f4a708f81 req-cceb582c-d71b-4382-8c5c-7ffeec06b0f4 service nova] Creating event network-vif-plugged:b6d5a291-fe8f-4e93-8b5f-7dfa9448f2b4 for instance 02806a84-8467-4b1c-9e5d-9044627225c8 on hv-cinder-maste Sep 25 01:10:15.477135 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-f87a0c66-bd64-41d7-9b05-392f4a708f81 req-cceb582c-d71b-4382-8c5c-7ffeec06b0f4 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041616 Sep 25 01:10:15.479023 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 555/1098] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:10:15 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:10:15.565964 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:15.576724 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-7f454f50-9d99-43ae-80af-903998d67c27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1318 microversion: 2.1 time: 0.240210 Sep 25 01:10:15.578115 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 544/1099] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:15 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1318 bytes in 242 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:10:16.598847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:16.599956 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=6 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:16.606102 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:16.606967 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:16.615351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:16.615843 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:16.616286 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:16.617039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:16.617538 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:16.617966 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:16.788500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:16.800027 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9c9b8e55-ab32-44d9-8463-2c572f47327c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1318 microversion: 2.1 time: 0.201806 Sep 25 01:10:16.800989 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 556/1100] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:16 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1318 bytes in 204 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:10:17.828018 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:17.828018 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:17.831577 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:17.838606 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:17.839098 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:17.839535 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:17.840264 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:17.840711 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:17.841130 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:18.001402 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:18.010177 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-56e5206d-0262-4666-a9df-cdd5ae2138f1 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1508 microversion: 2.1 time: 0.187097 Sep 25 01:10:18.010830 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 545/1101] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:17 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1508 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:10:33.609094 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "2cab75a0-9529-425d-b7ae-413fb57d84e8", "device": "/dev/sdb"}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:10:33.612133 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:33.614910 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=7 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:33.615742 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=19 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:33.622809 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:33.623309 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:33.623751 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:33.624475 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:33.624926 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:33.625381 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:33.977093 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-491402c6-5f6c-4572-84eb-623e4886721c tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "POST /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.371613 Sep 25 01:10:33.978052 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 557/1102] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 01:10:33 2022] POST /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8/os-volume_attachments => generated 194 bytes in 374 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:10:58.116843 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:58.118412 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:58.120952 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=5 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:58.121753 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=30 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:58.128692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:58.129313 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:58.129898 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:58.130727 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:58.131234 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:58.131729 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:58.363303 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.objects.instance [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lazy-loading 'services' on Instance uuid 02806a84-8467-4b1c-9e5d-9044627225c8 {{(pid=120450) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:10:58.448056 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-38850346-6aea-446e-8a16-c0e379f3e6b9 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "DELETE /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8/os-volume_attachments/2cab75a0-9529-425d-b7ae-413fb57d84e8" status: 202 len: 0 microversion: 2.1 time: 0.335737 Sep 25 01:10:58.452050 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 546/1103] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:10:58 2022] DELETE /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8/os-volume_attachments/2cab75a0-9529-425d-b7ae-413fb57d84e8 => generated 0 bytes in 337 msecs (HTTP/1.1 202) 9 headers in 361 bytes (2 switches on core 0) Sep 25 01:10:59.591236 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:59.592012 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=11,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:10:59.596528 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:59.597134 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:59.605802 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:59.606300 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:59.606739 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:59.607459 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:59.607917 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:59.608345 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:59.637146 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Going to try to terminate instance {{(pid=120451) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:10:59.734064 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-af377fca-3242-4258-8ba5-0bfe1304107b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "DELETE /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 204 len: 0 microversion: 2.1 time: 0.143986 Sep 25 01:10:59.735215 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 558/1104] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:10:59 2022] DELETE /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 9 headers in 363 bytes (1 switches on core 0) Sep 25 01:10:59.747640 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:10:59.748822 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:10:59.756708 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:59.757248 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:59.758364 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:59.759926 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:10:59.760433 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:10:59.761053 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:10:59.931342 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:10:59.939834 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-972d194b-5dbe-4b33-96e6-9a76f91fdc75 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1514 microversion: 2.1 time: 0.197890 Sep 25 01:10:59.940488 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 547/1105] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:10:59 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1514 bytes in 200 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:00.965819 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:00.966488 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:00.973832 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:00.974329 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:00.974770 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:00.975514 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:00.975945 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:00.976402 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:01.201002 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:01.212183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-f3f8c5c3-c071-48a7-bb48-d443c64f191f tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1514 microversion: 2.1 time: 0.250764 Sep 25 01:11:01.212859 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 559/1106] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:00 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1514 bytes in 253 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:11:02.236375 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:02.237348 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:02.248520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:02.249440 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:02.250233 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:02.251545 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:02.252409 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:02.253181 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:02.381475 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:02.400051 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-c3ff9eec-f79b-4c1e-b8f7-1ebc3eb5b388 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 200 len: 1472 microversion: 2.1 time: 0.166385 Sep 25 01:11:02.409091 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 548/1107] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:02 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 1472 bytes in 177 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:02.463480 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-6287b703-0822-4be7-85c8-e7b0309dd1b1 req-e337e2f6-1044-4b8c-b711-db8ab8ff5312 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "02806a84-8467-4b1c-9e5d-9044627225c8", "name": "network-vif-deleted", "tag": "b6d5a291-fe8f-4e93-8b5f-7dfa9448f2b4"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:11:02.475163 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6287b703-0822-4be7-85c8-e7b0309dd1b1 req-e337e2f6-1044-4b8c-b711-db8ab8ff5312 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:02.475697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6287b703-0822-4be7-85c8-e7b0309dd1b1 req-e337e2f6-1044-4b8c-b711-db8ab8ff5312 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:02.476131 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-6287b703-0822-4be7-85c8-e7b0309dd1b1 req-e337e2f6-1044-4b8c-b711-db8ab8ff5312 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:02.490229 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-6287b703-0822-4be7-85c8-e7b0309dd1b1 req-e337e2f6-1044-4b8c-b711-db8ab8ff5312 service nova] Creating event network-vif-deleted:b6d5a291-fe8f-4e93-8b5f-7dfa9448f2b4 for instance 02806a84-8467-4b1c-9e5d-9044627225c8 on hv-cinder-maste Sep 25 01:11:02.497996 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-6287b703-0822-4be7-85c8-e7b0309dd1b1 req-e337e2f6-1044-4b8c-b711-db8ab8ff5312 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.037617 Sep 25 01:11:02.500065 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 560/1108] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:11:02 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 41 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:11:03.423692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:03.424379 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 02806a84-8467-4b1c-9e5d-9044627225c8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:03.431849 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:03.432337 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:03.432778 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:03.433542 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:03.433997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:03.434434 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:03.450947 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.wsgi [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] HTTP exception thrown: Instance 02806a84-8467-4b1c-9e5d-9044627225c8 could not be found. Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Request method failure captured: Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: request: GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 HTTP/1.1 Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept: application/json Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Accept-Encoding: identity Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Connection: close Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Length: 0 Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Content-Type: application/json Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Host: 10.222.0.52 Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Openstack-System-Scope: None Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: User-Agent: python-urllib3/1.26.12 Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Auth-Token: gAAAAABjL6psHLt5aLJkMT30Tr0SgcvLtgHLR7PQj3-BYQETU4_n6CtVCYy8Mp3ghSPw9fngS3H9iPY_64KmyDblWAh1uKqKfpEifu_3-kBw7sw2qFX3DU8-yuIUV_LYbY3P854OEt5vxjijjjC0KHxS6qd1hed96RslepPK-RgMwEyvCJrbF7Q Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Id: None Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Domain-Name: None Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Identity-Status: Confirmed Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Is-Admin-Project: True Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Id: default Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Domain-Name: Default Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Id: 1f0ea61e4de94ce083c572843a3f09fa Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1338018223 Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Role: member,reader Sep 25 01:11:03.452813 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Roles: member,reader Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_1f0ea61e4de94ce083c572843a3f09fa", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/1f0ea61e4de94ce083c572843a3f09fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1f0ea61e4de94ce083c572843a3f09fa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1f0ea61e4de94ce083c572843a3f09fa"}]}] Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant: tempest-VolumesSnapshotTestJSON-1338018223 Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Id: 1f0ea61e4de94ce083c572843a3f09fa Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1338018223 Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User: tempest-VolumesSnapshotTestJSON-1338018223-project Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Id: default Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Domain-Name: Default Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Id: d978fab9ac0a4094a7e1008337ed2af9 Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: X-User-Name: tempest-VolumesSnapshotTestJSON-1338018223-project Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: method: > Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: exception: Instance 02806a84-8467-4b1c-9e5d-9044627225c8 could not be found. Sep 25 01:11:03.455470 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: action_args: {'id': '02806a84-8467-4b1c-9e5d-9044627225c8'} Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 02806a84-8467-4b1c-9e5d-9044627225c8 could not be found. Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:11:03.457353 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 02806a84-8467-4b1c-9e5d-9044627225c8 could not be found. Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 02806a84-8467-4b1c-9e5d-9044627225c8 could not be found. Sep 25 01:11:03.460255 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: ERROR nova.api.openstack.wsgi Sep 25 01:11:03.462636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Returning 404 to user: Instance 02806a84-8467-4b1c-9e5d-9044627225c8 could not be found. {{(pid=120450) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:11:03.462636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-ed633f0e-a9bf-4695-b6d8-bae1cd6d0a27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8" status: 404 len: 111 microversion: 2.1 time: 0.034720 Sep 25 01:11:03.462636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 549/1109] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:03 2022] GET /compute/v2.1/servers/02806a84-8467-4b1c-9e5d-9044627225c8 => generated 111 bytes in 36 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0) Sep 25 01:11:11.428447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Action: 'create', calling method: >, body: {"server": {"name": "tempest-VolumesSnapshotTestJSON-instance-1830290899", "imageRef": "2f7677e4-b12f-4656-9e36-b5892e4a684f", "flavorRef": "452", "networks": [{"uuid": "567e0576-161f-495c-b882-012356a1e499"}]}} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:11:11.492307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.network.neutron [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] validate_networks() for [('567e0576-161f-495c-b882-012356a1e499', None, None, None, None, None)] {{(pid=120451) validate_networks /opt/stack/nova/nova/network/neutron.py:2623}} Sep 25 01:11:11.899658 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Flavor limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:343}} Sep 25 01:11:11.900085 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Image limits 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:347}} Sep 25 01:11:11.900563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Flavor pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:383}} Sep 25 01:11:11.900970 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Image pref 0:0:0 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:387}} Sep 25 01:11:11.901428 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.virt.hardware [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Chose sockets=0, cores=0, threads=0; limits were sockets=65536, cores=65536, threads=65536 {{(pid=120451) get_cpu_topology_constraints /opt/stack/nova/nova/virt/hardware.py:425}} Sep 25 01:11:11.912482 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "00000000-0000-0000-0000-000000000000" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:11.913084 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "00000000-0000-0000-0000-000000000000" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:11.913728 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "00000000-0000-0000-0000-000000000000" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:11.914574 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:11.915318 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:11.915806 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:11.936447 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Getting quotas for project 1f0ea61e4de94ce083c572843a3f09fa. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:393}} Sep 25 01:11:11.941851 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.quota [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Getting quotas for user d978fab9ac0a4094a7e1008337ed2af9 and project 1f0ea61e4de94ce083c572843a3f09fa. Resources: {'ram', 'cores', 'instances'} {{(pid=120451) _get_quotas /opt/stack/nova/nova/quota.py:383}} Sep 25 01:11:11.956392 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Going to run 1 instances... {{(pid=120451) _provision_instances /opt/stack/nova/nova/compute/api.py:1403}} Sep 25 01:11:11.958891 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] block_device_mapping [BlockDeviceMapping(attachment_id=,boot_index=0,connection_info=None,created_at=,delete_on_termination=True,deleted=,deleted_at=,destination_type='local',device_name=None,device_type='disk',disk_bus=None,encrypted=False,encryption_format=None,encryption_options=None,encryption_secret_uuid=None,guest_format=None,id=,image_id='2f7677e4-b12f-4656-9e36-b5892e4a684f',instance=,instance_uuid=,no_device=False,snapshot_id=None,source_type='image',tag=None,updated_at=,uuid=,volume_id=None,volume_size=None,volume_type=None)] {{(pid=120451) _bdm_validate_set_size_and_instance /opt/stack/nova/nova/compute/api.py:1864}} Sep 25 01:11:12.011828 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-bed50c71-56a3-447b-9b16-058f821a755a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "POST /compute/v2.1/servers" status: 202 len: 380 microversion: 2.1 time: 0.588580 Sep 25 01:11:12.012938 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 561/1110] 10.222.0.52 () {60 vars in 1147 bytes} [Sun Sep 25 01:11:11 2022] POST /compute/v2.1/servers => generated 380 bytes in 590 msecs (HTTP/1.1 202) 10 headers in 451 bytes (1 switches on core 0) Sep 25 01:11:12.025400 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-6f7ce3d1-49c3-45cc-8130-62abd3999e78 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:12.026454 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-6f7ce3d1-49c3-45cc-8130-62abd3999e78 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:12.100572 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-6f7ce3d1-49c3-45cc-8130-62abd3999e78 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:12.109996 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-6f7ce3d1-49c3-45cc-8130-62abd3999e78 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1219 microversion: 2.1 time: 0.088833 Sep 25 01:11:12.111320 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 550/1111] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:12 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1219 bytes in 91 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:11:13.134777 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:13.135452 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:13.146841 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:13.147950 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:13.150123 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:13.151653 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:13.152189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:13.152687 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:13.224691 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:13.234279 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-c6ae5085-2709-4760-bfad-4a9620bc3608 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1271 microversion: 2.1 time: 0.103797 Sep 25 01:11:13.235187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 562/1112] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:13 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1271 bytes in 105 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:14.261839 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:14.262653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:14.265286 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=23,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:14.274923 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:14.275564 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:14.276116 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:14.277046 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:14.277686 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:14.278242 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:14.375946 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:14.386818 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5fb8114e-2e6b-4cd7-b3ad-e97e8b0e6767 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1277 microversion: 2.1 time: 0.130862 Sep 25 01:11:14.387415 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 551/1113] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:14 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1277 bytes in 132 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:11:15.428684 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:15.430533 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:15.440086 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:15.440563 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:15.441020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:15.441791 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:15.442249 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:15.442908 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:15.519061 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-8ecf077d-8e9b-4706-88e8-06c4e24c02a2 req-8d86db8a-da19-490a-9c86-91f1b3bd9e80 service nova] Action: 'create', calling method: >, body: {"events": [{"name": "network-changed", "server_uuid": "578132fa-c1d7-4ce3-a944-cd3c71c8a4e8", "tag": "ac18cb42-0ea0-4b75-a3e1-345296f02558"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:11:15.539085 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-8ecf077d-8e9b-4706-88e8-06c4e24c02a2 req-8d86db8a-da19-490a-9c86-91f1b3bd9e80 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:15.540309 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-8ecf077d-8e9b-4706-88e8-06c4e24c02a2 req-8d86db8a-da19-490a-9c86-91f1b3bd9e80 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.002s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:15.541508 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-8ecf077d-8e9b-4706-88e8-06c4e24c02a2 req-8d86db8a-da19-490a-9c86-91f1b3bd9e80 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:15.559527 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-8ecf077d-8e9b-4706-88e8-06c4e24c02a2 req-8d86db8a-da19-490a-9c86-91f1b3bd9e80 service nova] Creating event network-changed:ac18cb42-0ea0-4b75-a3e1-345296f02558 for instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 on hv-cinder-maste Sep 25 01:11:15.568687 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-8ecf077d-8e9b-4706-88e8-06c4e24c02a2 req-8d86db8a-da19-490a-9c86-91f1b3bd9e80 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 179 microversion: 2.1 time: 0.053739 Sep 25 01:11:15.570717 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 552/1114] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:11:15 2022] POST /compute/v2.1/os-server-external-events => generated 179 bytes in 56 msecs (HTTP/1.1 200) 9 headers in 357 bytes (2 switches on core 0) Sep 25 01:11:15.620171 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:15.629582 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-d3a3cf55-b5cf-4b9f-a8b1-e76a90a9c76a tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1319 microversion: 2.1 time: 0.220498 Sep 25 01:11:15.630585 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 563/1115] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:15 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1319 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:16.656878 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:16.657723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:16.666131 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:16.666682 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:16.667162 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:16.667935 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:16.668444 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:16.669007 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:16.829663 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:16.838583 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a2c8b96d-e6a7-4b0c-9b99-45496c766544 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1319 microversion: 2.1 time: 0.187123 Sep 25 01:11:16.839182 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 553/1116] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:16 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1319 bytes in 189 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:11:17.862305 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:17.863063 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:17.871669 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:17.872161 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:17.872594 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:17.873351 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:17.873802 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:17.874225 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:18.020495 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:18.029232 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-e799a028-c51f-4787-925f-321c19be6ea6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1319 microversion: 2.1 time: 0.170730 Sep 25 01:11:18.029841 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 564/1117] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:17 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1319 bytes in 172 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:19.055539 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:19.056141 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:19.063474 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:19.064003 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:19.064423 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:19.065152 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:19.065653 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:19.066088 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:19.219225 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:19.227697 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-5fc22373-cdc7-4945-b908-21d9bbb69e82 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1319 microversion: 2.1 time: 0.177324 Sep 25 01:11:19.228341 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 554/1118] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:19 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1319 bytes in 179 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:20.252571 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:20.253453 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:20.261106 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:20.261644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:20.262091 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:20.262808 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:20.263271 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:20.263733 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:20.432380 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:20.442704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9be88f9f-d8f2-445f-a36b-625e2b54f949 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1319 microversion: 2.1 time: 0.193849 Sep 25 01:11:20.443699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 565/1119] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:20 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1319 bytes in 196 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:21.468243 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:21.469076 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:21.476910 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:21.477463 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:21.477907 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:21.478636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:21.479096 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:21.479532 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:21.674669 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:21.685723 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-083f45a7-5570-4358-81d9-a6e8ded9ada0 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1319 microversion: 2.1 time: 0.221443 Sep 25 01:11:21.686325 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 555/1120] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:21 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1319 bytes in 223 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:21.782187 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [req-0ac330e0-b6eb-48cb-8a34-87cdb73b46d2 req-1a5f1faa-9906-4538-a936-eb7bc745ddbf service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "578132fa-c1d7-4ce3-a944-cd3c71c8a4e8", "name": "network-vif-plugged", "status": "completed", "tag": "ac18cb42-0ea0-4b75-a3e1-345296f02558"}]} {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:11:21.787933 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 25 01:11:21.788183 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Sun Sep 25 01:11:21 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/os-server-external-events (ip 10.222.0.52) !!! Sep 25 01:11:21.797021 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-0ac330e0-b6eb-48cb-8a34-87cdb73b46d2 req-1a5f1faa-9906-4538-a936-eb7bc745ddbf service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:21.797539 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-0ac330e0-b6eb-48cb-8a34-87cdb73b46d2 req-1a5f1faa-9906-4538-a936-eb7bc745ddbf service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:21.798001 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [req-0ac330e0-b6eb-48cb-8a34-87cdb73b46d2 req-1a5f1faa-9906-4538-a936-eb7bc745ddbf service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:21.812797 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.compute.server_external_events [req-0ac330e0-b6eb-48cb-8a34-87cdb73b46d2 req-1a5f1faa-9906-4538-a936-eb7bc745ddbf service nova] Creating event network-vif-plugged:ac18cb42-0ea0-4b75-a3e1-345296f02558 for instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 on hv-cinder-maste Sep 25 01:11:21.820607 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [req-0ac330e0-b6eb-48cb-8a34-87cdb73b46d2 req-1a5f1faa-9906-4538-a936-eb7bc745ddbf service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.041622 Sep 25 01:11:21.822155 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 566/1121] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:11:21 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 45 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:11:22.725911 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:22.726894 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:22.730489 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO oslo.messaging._drivers.impl_rabbit [-] A recoverable connection/channel error occurred, trying to reconnect: [Errno 104] Connection reset by peer Sep 25 01:11:22.731036 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: Sun Sep 25 01:11:22 2022 - SIGPIPE: writing to a closed pipe/socket/fd (probably the client disconnected) on request /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 (ip 10.222.0.52) !!! Sep 25 01:11:22.733692 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell0:SELECT=2 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:22.747160 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:22.747816 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:22.748337 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:22.749373 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:22.754710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:22.754710 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:22.932137 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:22.948189 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-999a9d8d-e292-4249-8650-6e49674a7993 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1319 microversion: 2.1 time: 0.228996 Sep 25 01:11:22.949241 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 556/1122] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:22 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1319 bytes in 231 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:23.987216 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=2 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:23.988166 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell0:SELECT=6 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:23.994455 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:23.995307 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:24.004506 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:24.005094 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:24.005644 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:24.006583 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:24.007787 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:24.008288 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:24.175262 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:24.184995 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-cb67bcad-990c-4cd2-9f09-dcbe06c377f6 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1508 microversion: 2.1 time: 0.198835 Sep 25 01:11:24.185659 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 567/1123] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:23 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1508 bytes in 201 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:24.211567 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Action: 'create', calling method: >, body: {"volumeAttachment": {"volumeId": "62dc3ed1-c7b8-4ffe-8ff3-b59214748285", "device": "/dev/sdb"}} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:11:24.214912 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:24.222357 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:24.222963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:24.223522 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:24.224447 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:24.225016 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:24.225646 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:24.550368 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-859d98fd-1130-4c8d-8870-14a1a394722b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "POST /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8/os-volume_attachments" status: 200 len: 194 microversion: 2.1 time: 0.342751 Sep 25 01:11:24.551015 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 557/1124] 10.222.0.52 () {60 vars in 1323 bytes} [Sun Sep 25 01:11:24 2022] POST /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8/os-volume_attachments => generated 194 bytes in 344 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:11:44.478697 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:44.479500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:44.481576 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_api:SELECT=36,nova_api:UPDATE=1,nova_api:INSERT=3 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:44.482399 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG dbcounter [-] [120451] Writing DB stats nova_cell1:SELECT=56,nova_cell1:UPDATE=1,nova_cell1:INSERT=1 {{(pid=120451) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:44.490344 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:44.490847 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:44.491301 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:44.492147 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:44.492608 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:44.493039 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:44.597812 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.objects.instance [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lazy-loading 'services' on Instance uuid 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 {{(pid=120451) obj_load_attr /opt/stack/nova/nova/objects/instance.py:1100}} Sep 25 01:11:44.683338 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-fe05f8db-e361-4f35-ad0e-aff7de244b6d tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "DELETE /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8/os-volume_attachments/62dc3ed1-c7b8-4ffe-8ff3-b59214748285" status: 202 len: 0 microversion: 2.1 time: 0.209136 Sep 25 01:11:44.684055 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 568/1125] 10.222.0.52 () {58 vars in 1416 bytes} [Sun Sep 25 01:11:44 2022] DELETE /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8/os-volume_attachments/62dc3ed1-c7b8-4ffe-8ff3-b59214748285 => generated 0 bytes in 211 msecs (HTTP/1.1 202) 9 headers in 361 bytes (1 switches on core 0) Sep 25 01:11:45.825763 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_api:SELECT=26 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:45.826680 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG dbcounter [-] [120450] Writing DB stats nova_cell1:SELECT=43,nova_cell1:UPDATE=2,nova_cell1:INSERT=1 {{(pid=120450) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 25 01:11:45.831309 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:45.831970 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:45.841963 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:45.842520 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:45.842997 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:45.843768 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:45.844271 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:45.844773 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:45.875133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Going to try to terminate instance {{(pid=120450) delete /opt/stack/nova/nova/compute/api.py:2719}} Sep 25 01:11:45.974410 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-b6043213-6a89-471c-a621-a90ce171df0b tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "DELETE /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 204 len: 0 microversion: 2.1 time: 0.148770 Sep 25 01:11:45.975087 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 558/1126] 10.222.0.52 () {58 vars in 1239 bytes} [Sun Sep 25 01:11:45 2022] DELETE /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 9 headers in 363 bytes (2 switches on core 0) Sep 25 01:11:45.985500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:45.986293 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:45.994300 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:45.994807 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:45.995260 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:45.996020 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:45.996495 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:45.996923 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:46.149741 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:46.158189 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-6acf3deb-8a77-4cf5-acc4-c226fac0a8a7 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1514 microversion: 2.1 time: 0.177892 Sep 25 01:11:46.158825 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 569/1127] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:45 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1514 bytes in 180 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:11:47.183892 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:47.184648 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.compute.api [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120450) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:47.191448 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:47.191938 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:47.192383 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:47.193109 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:47.193593 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:47.194524 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:47.350033 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.policy [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120450) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:47.361798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [None req-a485729a-bde1-4645-826e-0fc92de4ce27 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1514 microversion: 2.1 time: 0.182802 Sep 25 01:11:47.362750 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 559/1128] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:47 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1514 bytes in 185 msecs (HTTP/1.1 200) 9 headers in 358 bytes (1 switches on core 0) Sep 25 01:11:48.389359 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:48.390699 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:48.400062 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:48.400686 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:48.401292 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:48.402218 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:48.402780 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:48.403304 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:48.499626 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.policy [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Policy check for os_compute_api:os-extended-server-attributes failed with credentials {'is_admin': False, 'user_id': 'd978fab9ac0a4094a7e1008337ed2af9', 'user_domain_id': 'default', 'system_scope': None, 'domain_id': None, 'project_id': '1f0ea61e4de94ce083c572843a3f09fa', 'project_domain_id': 'default', 'roles': ['member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=120451) authorize /opt/stack/nova/nova/policy.py:199}} Sep 25 01:11:48.510407 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-3435d546-f782-43df-bb34-a8f2a623baad tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 200 len: 1472 microversion: 2.1 time: 0.125802 Sep 25 01:11:48.512143 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 570/1129] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:48 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 1472 bytes in 128 msecs (HTTP/1.1 200) 9 headers in 358 bytes (2 switches on core 0) Sep 25 01:11:48.600798 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG nova.api.openstack.wsgi [req-c296730c-65e6-4f08-a116-be92d580b09e req-93a5b4ed-6f25-4aee-a98a-f06b8450e9d8 service nova] Action: 'create', calling method: >, body: {"events": [{"server_uuid": "578132fa-c1d7-4ce3-a944-cd3c71c8a4e8", "name": "network-vif-deleted", "tag": "ac18cb42-0ea0-4b75-a3e1-345296f02558"}]} {{(pid=120450) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:511}} Sep 25 01:11:48.614133 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c296730c-65e6-4f08-a116-be92d580b09e req-93a5b4ed-6f25-4aee-a98a-f06b8450e9d8 service nova] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:48.614770 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c296730c-65e6-4f08-a116-be92d580b09e req-93a5b4ed-6f25-4aee-a98a-f06b8450e9d8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:48.615636 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: DEBUG oslo_concurrency.lockutils [req-c296730c-65e6-4f08-a116-be92d580b09e req-93a5b4ed-6f25-4aee-a98a-f06b8450e9d8 service nova] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.001s {{(pid=120450) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:48.630500 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.compute.server_external_events [req-c296730c-65e6-4f08-a116-be92d580b09e req-93a5b4ed-6f25-4aee-a98a-f06b8450e9d8 service nova] Creating event network-vif-deleted:ac18cb42-0ea0-4b75-a3e1-345296f02558 for instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 on hv-cinder-maste Sep 25 01:11:48.637725 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: INFO nova.api.openstack.requestlog [req-c296730c-65e6-4f08-a116-be92d580b09e req-93a5b4ed-6f25-4aee-a98a-f06b8450e9d8 service nova] 10.222.0.52 "POST /compute/v2.1/os-server-external-events" status: 200 len: 183 microversion: 2.1 time: 0.133711 Sep 25 01:11:48.638607 dv-cinder-master-3-iscsi devstack@n-api.service[120450]: [pid: 120450|app: 0|req: 560/1130] 10.222.0.52 () {64 vars in 1317 bytes} [Sun Sep 25 01:11:48 2022] POST /compute/v2.1/os-server-external-events => generated 183 bytes in 136 msecs (HTTP/1.1 200) 9 headers in 357 bytes (1 switches on core 0) Sep 25 01:11:49.548784 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Calling method '>' {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:513}} Sep 25 01:11:49.549500 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.compute.api [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] [instance: 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8] Fetching instance by UUID {{(pid=120451) get /opt/stack/nova/nova/compute/api.py:2975}} Sep 25 01:11:49.556804 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:49.557324 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:49.557769 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:49.558704 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Acquiring lock "ade94c2f-42bc-4402-b847-004397ecdfd4" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:404}} Sep 25 01:11:49.559259 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" acquired by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: waited 0.001s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:409}} Sep 25 01:11:49.559799 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG oslo_concurrency.lockutils [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Lock "ade94c2f-42bc-4402-b847-004397ecdfd4" "released" by "nova.context.set_target_cell..get_or_set_cached_cell_and_set_connections" :: held 0.000s {{(pid=120451) inner /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:423}} Sep 25 01:11:49.576773 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.wsgi [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] HTTP exception thrown: Instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 could not be found. Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Request method failure captured: Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: request: GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 HTTP/1.1 Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept: application/json Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Accept-Encoding: identity Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Connection: close Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Length: 0 Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Content-Type: application/json Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Host: 10.222.0.52 Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: Openstack-System-Scope: None Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: User-Agent: python-urllib3/1.26.12 Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Auth-Token: gAAAAABjL6psHLt5aLJkMT30Tr0SgcvLtgHLR7PQj3-BYQETU4_n6CtVCYy8Mp3ghSPw9fngS3H9iPY_64KmyDblWAh1uKqKfpEifu_3-kBw7sw2qFX3DU8-yuIUV_LYbY3P854OEt5vxjijjjC0KHxS6qd1hed96RslepPK-RgMwEyvCJrbF7Q Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Id: None Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Domain-Name: None Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Identity-Status: Confirmed Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Is-Admin-Project: True Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Id: default Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Domain-Name: Default Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Id: 1f0ea61e4de94ce083c572843a3f09fa Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Project-Name: tempest-VolumesSnapshotTestJSON-1338018223 Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Role: member,reader Sep 25 01:11:49.578623 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Roles: member,reader Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Service-Catalog: [{"type": "object-store", "name": "swift", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:8080/v1/AUTH_1f0ea61e4de94ce083c572843a3f09fa", "adminURL": "http://10.222.0.52:8080"}]}, {"type": "compute", "name": "nova", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2.1"}]}, {"type": "compute_legacy", "name": "nova_legacy", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/compute/v2/1f0ea61e4de94ce083c572843a3f09fa"}]}, {"type": "network", "name": "neutron", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52:9696/networking"}]}, {"type": "volumev3", "name": "cinderv3", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1f0ea61e4de94ce083c572843a3f09fa"}]}, {"type": "image", "name": "glance", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/image"}]}, {"type": "placement", "name": "placement", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/placement"}]}, {"type": "identity", "name": "keystone", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/identity"}]}, {"type": "block-storage", "name": "cinder", "endpoints": [{"region": "RegionOne", "publicURL": "http://10.222.0.52/volume/v3/1f0ea61e4de94ce083c572843a3f09fa"}]}] Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant: tempest-VolumesSnapshotTestJSON-1338018223 Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Id: 1f0ea61e4de94ce083c572843a3f09fa Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-Tenant-Name: tempest-VolumesSnapshotTestJSON-1338018223 Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User: tempest-VolumesSnapshotTestJSON-1338018223-project Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Id: default Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Domain-Name: Default Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Id: d978fab9ac0a4094a7e1008337ed2af9 Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: X-User-Name: tempest-VolumesSnapshotTestJSON-1338018223-project Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: method: > Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: exception: Instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 could not be found. Sep 25 01:11:49.580952 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: action_args: {'id': '578132fa-c1d7-4ce3-a944-cd3c71c8a4e8'} Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: {{(pid=120451) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:541}} Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 468, in get_instance Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return compute_api.get(context, instance_id, Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2978, in get Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance(context, instance_id, Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2925, in _get_instance Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_instance_from_cell(context, inst_map, Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/compute/api.py", line 2882, in _get_instance_from_cell Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exception.InstanceNotFound(instance_id=uuid) Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.exception.InstanceNotFound: Instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 could not be found. Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 630, in dispatch Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return method(req=request, **action_args) Sep 25 01:11:49.582740 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 664, in wrapped Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi return f(*args, **kwargs) Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 470, in show Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = self._get_server( Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/compute/servers.py", line 376, in _get_server Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi instance = common.get_instance(self.compute_api, context, Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/common.py", line 472, in get_instance Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise exc.HTTPNotFound(explanation=e.format_message()) Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi webob.exc.HTTPNotFound: Instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 could not be found. Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi During handling of the above exception, another exception occurred: Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Traceback (most recent call last): Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 539, in _process_stack Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi action_result = self.dispatch(meth, request, action_args) Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi File "/opt/stack/nova/nova/api/openstack/wsgi.py", line 372, in __exit__ Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi raise Fault(ex_value) Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi nova.api.openstack.wsgi.Fault: Instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 could not be found. Sep 25 01:11:49.586492 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: ERROR nova.api.openstack.wsgi Sep 25 01:11:49.588882 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: DEBUG nova.api.openstack.wsgi [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] Returning 404 to user: Instance 578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 could not be found. {{(pid=120451) __call__ /opt/stack/nova/nova/api/openstack/wsgi.py:942}} Sep 25 01:11:49.588882 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: INFO nova.api.openstack.requestlog [None req-9f09fa4d-2a66-4349-91fe-5713f7d16697 tempest-VolumesSnapshotTestJSON-1338018223 tempest-VolumesSnapshotTestJSON-1338018223-project] 10.222.0.52 "GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8" status: 404 len: 111 microversion: 2.1 time: 0.035568 Sep 25 01:11:49.588882 dv-cinder-master-3-iscsi devstack@n-api.service[120451]: [pid: 120451|app: 0|req: 571/1131] 10.222.0.52 () {58 vars in 1236 bytes} [Sun Sep 25 01:11:49 2022] GET /compute/v2.1/servers/578132fa-c1d7-4ce3-a944-cd3c71c8a4e8 => generated 111 bytes in 37 msecs (HTTP/1.1 404) 9 headers in 379 bytes (1 switches on core 0)